-- Logs begin at Thu 2022-06-30 11:42:04 UTC, end at Thu 2022-06-30 12:50:58 UTC. -- Jun 30 12:18:39.262262 dv-cinder-846146-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 12:18:39.271281 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 12:18:39 2022] *** Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: nodename: dv-cinder-846146-5-iscsi Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: machine: x86_64 Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: clock source: unix Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: pcre jit disabled Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: detected number of CPU cores: 4 Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: current working directory: / Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: detected binary path: /usr/bin/uwsgi-core Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: your processes number limit is 47680 Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: your memory page size is 4096 bytes Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: detected max file descriptor number: 2048 Jun 30 12:18:39.273276 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: lock engine: pthread robust mutexes Jun 30 12:18:39.273990 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: thunder lock: enabled Jun 30 12:18:39.273990 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 12:18:39.273990 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 12:18:39.330265 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: Python main interpreter initialized at 0x5572827c39c0 Jun 30 12:18:39.330265 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: python threads support enabled Jun 30 12:18:39.330265 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: your server socket listen backlog is limited to 100 connections Jun 30 12:18:39.330265 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: your mercy for graceful operations on workers is 90 seconds Jun 30 12:18:39.330708 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 12:18:39.330843 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: *** Operational MODE: preforking *** Jun 30 12:18:39.331248 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: *** uWSGI is running in multiple interpreter mode *** Jun 30 12:18:39.331248 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: spawned uWSGI master process (pid: 80085) Jun 30 12:18:39.331501 dv-cinder-846146-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 12:18:39.335591 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: spawned uWSGI worker 1 (pid: 80096, cores: 1) Jun 30 12:18:39.335591 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: spawned uWSGI worker 2 (pid: 80097, cores: 1) Jun 30 12:18:39.335591 dv-cinder-846146-5-iscsi devstack@keystone.service[80085]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 12:18:44.812607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:18:44.812607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: from cryptography.utils import int_from_bytes Jun 30 12:18:44.813464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:18:44.813464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: from cryptography.utils import int_from_bytes Jun 30 12:18:44.815451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:18:44.815451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: from cryptography.utils import int_from_bytes Jun 30 12:18:44.815942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:18:44.815942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: from cryptography.utils import int_from_bytes Jun 30 12:18:44.862051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 12:18:44.862249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 12:18:44.862411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] command line args: [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 12:18:44.862650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 12:18:44.862854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ================================================================================ {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 12:18:44.862991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 12:18:44.863079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] admin_token = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.863147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 12:18:44.863293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] config_dir = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.863380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] command line args: [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 12:18:44.863515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.863651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 12:18:44.863745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] config_source = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.863820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ================================================================================ {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 12:18:44.863896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] debug = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.863973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] admin_token = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] config_dir = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] config_source = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] debug = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.864992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_config_append = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_dir = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.865886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_file = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_config_append = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_dir = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_file = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.866979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.867938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868763 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.868978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] publish_errors = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.869923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] publish_errors = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.870948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] standard_threads = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.871975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] use_journal = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] standard_threads = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] use_json = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] use_stderr = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] use_syslog = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.872982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.873081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] use_journal = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.873160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] use_json = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.873231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.873309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] use_stderr = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.873473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.873563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] use_syslog = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.873700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:18:44.873782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.873946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.874987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.875938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.876915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.877857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.878887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.879862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.880974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.881998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.882933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.883857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.884944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.885891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.886901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.887813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.888950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.889990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.890946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.891912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.892992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.893994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894380 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.894905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.895959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.896982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.external = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.897921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.password = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] auth.token = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.898888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.external = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.password = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.899974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] auth.token = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900299 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.900924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.901964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.902917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.903982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.904956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.905919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.906855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.907963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.908995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.909921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.910983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.911938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.912930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.913997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.914874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.915961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.916995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.917916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.918863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.919986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.920945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.921941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.922862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.923953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.924994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.925922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.926862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.927950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.928997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.929944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] role.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.930862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] role.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] role.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.931968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] role.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.932947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.933994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.934985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.935974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.936958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.937896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.938981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.939991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.940964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.941115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.941194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.941359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.941436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.941593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.941674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.942995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.943926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.connection = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.944913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.connection = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945392 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.945856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.946937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.947949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.948093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.948171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.948263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 12:18:44.948342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.948551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.948806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.949045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.949279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:18:44.949448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 12:18:44.978821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.979003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.979253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.979826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.980055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.980311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.987829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:44.988742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.015762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.015966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.016504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.016763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.161219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.161930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.211255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.211413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.211513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.211587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.274551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.274751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.274993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.275943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.276151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.299818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.300928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.301007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.301127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.301207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.301401 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.301635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.312257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.312618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.325273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.325519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.325710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.325823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80096) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.326839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.327056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.327273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80097) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:18:45.343966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.344157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:18:45.344511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.344657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.344765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:18:45.344856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:18:45.344993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.345159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.345254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.345330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:18:45.345484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.345606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.345719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.345917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.346135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.346380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.356477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.356834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.357100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.357295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.357420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.357542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.357836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.358111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.365578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.366324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.368854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.369129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.369404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.369590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.369686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.369827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.369974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.370997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.371120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.371263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.371390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.371509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.371634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.371906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.372169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.397729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.399320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.399512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.399607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.401045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.401324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.404900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.405174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.405564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.406666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.406942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.407216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.412949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.413176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.414653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.414908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.417619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.417905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.418177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.418457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.418723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.418992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.419231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.419757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.419852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.419986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.419986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.420201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.420469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.420767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.421021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.421281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.438101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.438364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.439820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.440103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.443379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.445100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.447012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.447298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.448735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.449025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.452441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.454211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.456036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.457816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.459686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.459980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.460264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.460546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.460857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.461267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.461730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.462027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.462304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.462583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.462863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.463158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.480097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.480375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.480647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.480917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.482711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.482993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.483261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.483524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.488812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.490502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.490772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.491390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.493105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.493377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.494027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.494301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.494578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.494850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.495117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.496604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.496910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.497193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.497481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.497757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.508319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.508587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.508883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.509156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.509440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.510973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.511109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.511246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.511324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.511511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.511799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.512075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.512341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.512619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.529275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.531750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.532511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.535004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.535807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.537569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.538337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.539179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.539455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.539732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.539932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.541711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.541989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.542273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.547071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.548669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.549630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.551217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.551880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.552181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.552569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.552871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.554474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.554761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.555039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.555317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.562140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.562431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.562707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.564505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.564815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.565093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.570871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80096) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.571156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.571430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.571697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.571966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.572610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.572922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.573200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.573324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80097) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:18:45.573480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.573608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.573772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.573909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80096) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.574878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.575086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.575516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.575782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.576054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.576316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.576586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.576881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80097) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:18:45.612365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572827c39c0 pid: 80096 (default app) Jun 30 12:18:45.614759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572827c39c0 pid: 80097 (default app) Jun 30 12:18:45.615788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12bb76-0781-4662-8ca9-c53c5274e72a None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:45.616081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12bb76-0781-4662-8ca9-c53c5274e72a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:45.616348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12bb76-0781-4662-8ca9-c53c5274e72a None None] PATH_INFO: `/v3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:45.620612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu Jun 30 12:18:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:18:53.390526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:53.391419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:53.391419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:53.398675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:18:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:18:53.406992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:53.407432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:53.407524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:53.476412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: INFO dbcounter [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] Registered counter for database keystone Jun 30 12:18:53.500280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG oslo_db.sqlalchemy.engines [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80096) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 12:18:53.507864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writer thread running {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 12:18:53.688796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG passlib.handlers.bcrypt [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80096) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 12:18:53.689230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG passlib.handlers.bcrypt [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80096) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 12:18:53.745633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:18:53.747146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:18:53.889952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:18:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 12:18:54.095538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:54.178108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: INFO dbcounter [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] Registered counter for database keystone Jun 30 12:18:54.200282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG oslo_db.sqlalchemy.engines [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80097) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 12:18:54.210249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writer thread running {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 12:18:54.333268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a42a4-9e46-4af2-a7cf-ac7848f897e5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:54.338140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:54.338815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:54.339102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:54.339378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:54.348064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:54.400002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 12:18:54.400617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35302-920a-4476-b14a-efb1abd638d6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:54.411349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu Jun 30 12:18:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:18:54.418479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:54.441278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b3be9-6566-4ce6-85c3-65e0ee3b468d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:54.448195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:54.449366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:54.449650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:54.449924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] PATH_INFO: `/v3/projects/e7a9e0309e534d4980ce851f75758fe8` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:54.454399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] RBAC: Authorizing `identity:get_project(project_id=e7a9e0309e534d4980ce851f75758fe8)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:54.505390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 12:18:54.505725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebdd8c-64a3-4bb5-a602-138c7360c38a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:54.510659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu Jun 30 12:18:54 2022] GET /identity/v3/projects/e7a9e0309e534d4980ce851f75758fe8 => generated 344 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:18:56.513057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d45eb-69ab-4616-934e-1ece32dc5574 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:56.513929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d45eb-69ab-4616-934e-1ece32dc5574 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:56.513929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d45eb-69ab-4616-934e-1ece32dc5574 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:56.521051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:18:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:18:56.526667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:56.527099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:56.527193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:56.550099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:18:56.551856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:18:56.561343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:18:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 12:18:56.584651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d45eb-69ab-4616-934e-1ece32dc5574 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:56.599675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d45eb-69ab-4616-934e-1ece32dc5574 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:56.606927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b66dc-f4e8-4e78-ab45-b7f405d1b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:56.607840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b66dc-f4e8-4e78-ab45-b7f405d1b12f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:56.608179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b66dc-f4e8-4e78-ab45-b7f405d1b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:56.608580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b66dc-f4e8-4e78-ab45-b7f405d1b12f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:56.622326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b66dc-f4e8-4e78-ab45-b7f405d1b12f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:56.625156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b66dc-f4e8-4e78-ab45-b7f405d1b12f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:56.635052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu Jun 30 12:18:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:18:56.641226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:56.655865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875841f-ca0b-401c-b52b-5d9efae391af None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:56.660755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9166ab-6fbe-4426-9fcb-bc97aaf730e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:56.661628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9166ab-6fbe-4426-9fcb-bc97aaf730e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:56.661909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9166ab-6fbe-4426-9fcb-bc97aaf730e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:56.662184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9166ab-6fbe-4426-9fcb-bc97aaf730e5 admin admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:56.663791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9166ab-6fbe-4426-9fcb-bc97aaf730e5 admin admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:56.665695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9166ab-6fbe-4426-9fcb-bc97aaf730e5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:56.667289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu Jun 30 12:18:56 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:18:58.773674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30ef4-dbb4-4038-9ec7-d0d1def65797 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:58.774229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30ef4-dbb4-4038-9ec7-d0d1def65797 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:58.774229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c30ef4-dbb4-4038-9ec7-d0d1def65797 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:58.775570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:18:58.781129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:58.781529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:58.781730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:58.783945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb5af2-8e60-42b5-b1d1-25dbb062ea41 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:58.784228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb5af2-8e60-42b5-b1d1-25dbb062ea41 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:58.784509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb5af2-8e60-42b5-b1d1-25dbb062ea41 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:58.785791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:18:58.790723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:58.791009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:58.791286 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:58.799745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:18:58.801025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:18:58.831622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG passlib.handlers.bcrypt [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80097) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 12:18:58.832038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG passlib.handlers.bcrypt [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80097) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 12:18:58.877190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:18:58.878807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:18:58.885668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:18:58.909397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:58.934673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:58.982727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:18:59.004959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.016571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279df20-66ba-45a5-baee-fdf99ff2c4d2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.052142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.052920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.053204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.053478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.064651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:18:59.076244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:18:59.077206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:18:59.078163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:18:59.083784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.105848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb339a-d05b-49d9-8687-ffa71ecbf70f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.108106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:59.146662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.147807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.148086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.148359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.164544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:59.186368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1960da-8743-4229-92a5-8ce68d30b3c6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.220444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.222026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.222283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.222957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.223241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.223518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.224269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:18:59.224568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.225989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.238166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:18:59.243784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.320362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:18:59.325932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.328975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:59.343461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8137cde-74a7-43ae-ae08-5d35393bfca8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.378975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.379884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.380187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.380441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.381652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.383475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.385142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:18:59.390603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.404964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:59.422794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9e7be-27b3-4426-94cf-fb752a8da6d3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.455987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.456820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.457106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.457380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.475686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.477261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.491251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f0a07-768b-4335-9913-e9fc68ec44cd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:18:59.500046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:18:59.501458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:18:59.502240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:18:59.502240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:18:59.502240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:18:59.502240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:18:59.502240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:18:59.507143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.529350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.530157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.530435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.530707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.532369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.533896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.535510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:18:59.541296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:18:59.568981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:59.613412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef6696-2157-4af3-9827-08e24bf89e49 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.626013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:18:59.645010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f816a-673e-4833-9ac0-b3dd8b5b51d1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:18:59.648334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.649053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.649340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.649618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.650351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:18:59.650660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.652090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b506ba5-39c7-4e30-80f6-594edbb837c6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.669092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:18:59.678549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:18:59.679240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:18:59.679518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:18:59.679784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:18:59.680656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.domain.id=default, user.id=bfba4b29b3e04e04b163f6e8cff16484) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:18:59.680978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:18:59.682456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f87b0-d16b-45b7-8295-1edb01b92cf6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:18:59.690518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Thu Jun 30 12:18:59 2022] GET /identity/v3/role_assignments?user.id=bfba4b29b3e04e04b163f6e8cff16484&scope.domain.id=default&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:01.930860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c99052-b07f-4b53-8b27-5c8107dd3a5f None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:01.931406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c99052-b07f-4b53-8b27-5c8107dd3a5f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:01.931636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c99052-b07f-4b53-8b27-5c8107dd3a5f None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:01.932994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:01.938897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab5ae5-ccb1-4290-8770-1f259277271b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:01.939343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab5ae5-ccb1-4290-8770-1f259277271b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:01.939521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab5ae5-ccb1-4290-8770-1f259277271b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:01.960691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-bfab5ae5-ccb1-4290-8770-1f259277271b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:01.961919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-bfab5ae5-ccb1-4290-8770-1f259277271b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:02.047723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 12/23] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:02.082566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c99052-b07f-4b53-8b27-5c8107dd3a5f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:02.225316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c99052-b07f-4b53-8b27-5c8107dd3a5f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:02.246430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:02.268359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c5da7-514c-4d1c-8229-44fdecf66b57 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.268699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c5da7-514c-4d1c-8229-44fdecf66b57 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.268944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c5da7-514c-4d1c-8229-44fdecf66b57 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.270522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:02.274244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:02.275086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.275371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.275646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.282317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb8f2-5f20-4630-8f66-0f365988ea27 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.282610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb8f2-5f20-4630-8f66-0f365988ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.282892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb8f2-5f20-4630-8f66-0f365988ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.307591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-441fb8f2-5f20-4630-8f66-0f365988ea27 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:02.308816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-441fb8f2-5f20-4630-8f66-0f365988ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:02.321294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:02.334998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:02.335878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:02.336658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 12/25] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:02.345949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:02.456190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 14/26] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:02.478140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b607-e653-4e55-ab85-2f022fa667fc None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.478731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b607-e653-4e55-ab85-2f022fa667fc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.479212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165b607-e653-4e55-ab85-2f022fa667fc None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.484849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 15/27] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:02.495587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.496172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.496662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.509573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:02.527679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:02.536794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9173eff-7352-43cf-9d84-0ee579e6d252 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:02.536967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:02.565874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:02.566676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.566969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.567249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.568395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:02.568728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:02.570268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:02.583597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 13/28] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:02.584223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:02.667784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:02.680687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 16/29] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:02.681567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:02.788107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dee173-cb0c-420f-9281-33126f450417 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:02.825529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:02.826616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.827131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.827599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.839109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:02.840867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:02.841129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbd44c-c333-4903-b68b-acdfdcf4babb None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:19:02.859514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:02.860384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:02.861333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:19:02.861333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:19:02.861333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:19:02.861333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:02.863260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 14/30] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:02.869052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:02.881179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf480fa-45d2-407c-8b64-21e70582a3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:02.882080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf480fa-45d2-407c-8b64-21e70582a3ab None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.882764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf480fa-45d2-407c-8b64-21e70582a3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.883053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf480fa-45d2-407c-8b64-21e70582a3ab None admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.884550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf480fa-45d2-407c-8b64-21e70582a3ab None admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:02.886454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf480fa-45d2-407c-8b64-21e70582a3ab None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:02.892113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:02 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:02.892113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2b05d-d11e-468f-a27d-cb946c418cd7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:02.892113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2b05d-d11e-468f-a27d-cb946c418cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:02.892113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2b05d-d11e-468f-a27d-cb946c418cd7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:02.896121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 18/32] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:02.897018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2b05d-d11e-468f-a27d-cb946c418cd7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:02.900206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:02.987967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:03.002184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70989a-24ad-4d15-ae9f-d43604962908 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:03.018214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2b05d-d11e-468f-a27d-cb946c418cd7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:03.048883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c6709-cec4-4955-b32b-ffbdda2786df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:03.050021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c6709-cec4-4955-b32b-ffbdda2786df None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.050468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c6709-cec4-4955-b32b-ffbdda2786df None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.051086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c6709-cec4-4955-b32b-ffbdda2786df None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.052659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c6709-cec4-4955-b32b-ffbdda2786df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:03.054227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c6709-cec4-4955-b32b-ffbdda2786df None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:03.056658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:03.057451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.057832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.058206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.059045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:03.059471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:03.061264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:03.082646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 19/33] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:03.087495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:03.152016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:03.205428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7bfec-a020-4db8-9036-f8b37943cf88 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:03.217879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 15/34] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jun 30 12:19:02 2022] POST /identity/v3/roles => generated 225 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:03.226755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.227349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.227828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.251298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:03.253519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:03.254946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:03.255771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.256053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.256324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.260231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:03.261790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:03.268512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 20/35] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:03.269438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:03.331331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 16/36] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:03.332483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:03.440817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd02739-1d1d-4e3c-9adf-96f489a2206c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:03.450788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:03.478004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43523e3-f896-415f-ae01-9c765b15350f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:03.481314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:03.489179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7b24-defa-4641-8bd7-8a9d8ce286c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:03.489378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7b24-defa-4641-8bd7-8a9d8ce286c8 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.489768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7b24-defa-4641-8bd7-8a9d8ce286c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.489865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7b24-defa-4641-8bd7-8a9d8ce286c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.492661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7b24-defa-4641-8bd7-8a9d8ce286c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:03.494027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7b24-defa-4641-8bd7-8a9d8ce286c8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:03.529205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:03.530111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.531381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.531381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] PATH_INFO: `/v3/domains/default/users/bfba4b29b3e04e04b163f6e8cff16484/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.533250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bfba4b29b3e04e04b163f6e8cff16484, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:03.534877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:03.555307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 17/37] 10.222.0.57 () {58 vars in 1361 bytes} [Thu Jun 30 12:19:03 2022] PUT /identity/v3/domains/default/users/bfba4b29b3e04e04b163f6e8cff16484/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:03.556225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:03.637500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 21/38] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:19:03 2022] POST /identity/v3/projects => generated 301 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:03.646590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250c70-8c7f-4dfd-b9ae-6c05fb0cd642 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.646881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250c70-8c7f-4dfd-b9ae-6c05fb0cd642 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.647168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250c70-8c7f-4dfd-b9ae-6c05fb0cd642 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.649027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 22/39] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:03.650115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a68f-b996-44de-9f0f-e3b7c28df4e7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.650407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a68f-b996-44de-9f0f-e3b7c28df4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.650691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a68f-b996-44de-9f0f-e3b7c28df4e7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.652653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 23/40] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:03.660412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.660412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.660412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.679364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:03.681024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:03.689195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03095085-fae8-4a8c-9573-128b8ea268fe None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:03.731024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133093a9-b7f0-44db-8eb1-e5453eced20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:03.732057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133093a9-b7f0-44db-8eb1-e5453eced20c None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.732519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133093a9-b7f0-44db-8eb1-e5453eced20c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.733067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133093a9-b7f0-44db-8eb1-e5453eced20c None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.734161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133093a9-b7f0-44db-8eb1-e5453eced20c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:03.736526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133093a9-b7f0-44db-8eb1-e5453eced20c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:03.752523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:03.760198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 18/41] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jun 30 12:19:03 2022] POST /identity/v3/roles => generated 219 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:03.763168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:03.763543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:03.763861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:03.777100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 24/42] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:03.786457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:03.787720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:03.818288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:03.883404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 19/43] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:03.907641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:04.044170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e31187-819e-4bbf-8166-1f64af0521f3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:04.080672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:04.087198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:04.087897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:04.088173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:04.088660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:04.089305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:04.090953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bcfc7-21b7-4c6e-bc2a-6c3c63f15fc1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:04.091388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:04.105623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:04.106419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 20/44] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jun 30 12:19:03 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:04.133513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673339d-5c37-4004-b116-bb90e689b483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:04.134354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673339d-5c37-4004-b116-bb90e689b483 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:04.134641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673339d-5c37-4004-b116-bb90e689b483 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:04.134917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673339d-5c37-4004-b116-bb90e689b483 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:04.136248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673339d-5c37-4004-b116-bb90e689b483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:04.138066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673339d-5c37-4004-b116-bb90e689b483 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:04.140066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 25/45] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:04.145883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:04.264281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5f39-e648-4af1-ad4b-5e9b94d4da57 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:04.299610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373962b-3238-4147-a5ff-af0234217915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:04.300570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373962b-3238-4147-a5ff-af0234217915 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:04.300979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373962b-3238-4147-a5ff-af0234217915 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:04.301255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373962b-3238-4147-a5ff-af0234217915 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:04.302032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373962b-3238-4147-a5ff-af0234217915 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:04.303451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373962b-3238-4147-a5ff-af0234217915 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:04.332455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 21/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:19:04 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:05.743181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cef23-a71a-4fb4-a075-58efeb743778 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:05.743726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cef23-a71a-4fb4-a075-58efeb743778 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:05.743726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cef23-a71a-4fb4-a075-58efeb743778 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:05.745155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 26/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:05.750586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:05.750986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:05.751152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:05.769432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:05.770674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:05.810812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:05.846428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 22/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:05.871294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cef23-a71a-4fb4-a075-58efeb743778 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:05.983260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cef23-a71a-4fb4-a075-58efeb743778 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.019572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.020301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.020587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.020966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.031952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:06.039645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:06.040760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:06.041613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 27/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:06.046926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:06.258848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c2a0-511e-40a5-a675-8f08c3979ecd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.294011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.294622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.294906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.295187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.295954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:06.296266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:06.297756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2153dff-8e94-4b26-a51a-e366673dc238 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:06.306293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 23/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:06.312100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:06.339864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f109a-773d-487d-9fc1-b868e034ea3f None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.340157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f109a-773d-487d-9fc1-b868e034ea3f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.340441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f109a-773d-487d-9fc1-b868e034ea3f None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.341916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 24/51] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:06.346929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.347227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.347504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.355800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:06.364433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:06.365708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:06.392434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:06.415103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be43848-4981-4abc-9e73-a921d405747c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.432961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 25/52] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:06.448368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.449295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.449574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.449849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.451043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:06.452533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:06.454576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 28/53] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:06 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:06.456378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:06.460470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:06.563912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3612e29-a646-4b65-8dbc-6e613e9ea4d2 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.569592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc7ca-6f04-462f-958d-40f57cf7d8f0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.578172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:06.597784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.598616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.598905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.599180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.600357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:06.601866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:06.603468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.603583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 26/54] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:06.604276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.604562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.604884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.606077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:06.607544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:06.614263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 29/55] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:06.614410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:06.615230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:06.641219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:06.710485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1579-42f7-4838-8cec-a99ef3f5513c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.714965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a2855-03e9-4eeb-a00e-2eb2ad72dd6f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:06.746438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae14f1-b44b-4373-b89c-aca1243dbf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.747111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae14f1-b44b-4373-b89c-aca1243dbf20 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.747400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae14f1-b44b-4373-b89c-aca1243dbf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.747682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae14f1-b44b-4373-b89c-aca1243dbf20 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.748089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:06.748542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae14f1-b44b-4373-b89c-aca1243dbf20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:06.748769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:06.749051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:06.749326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:06.749894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae14f1-b44b-4373-b89c-aca1243dbf20 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:06.750161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.domain.id=default, user.id=bfba4b29b3e04e04b163f6e8cff16484) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:06.750570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:06.752254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7088962-0b09-4f8c-965b-944dc5260f21 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:06.758296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 30/56] 10.222.0.57 () {56 vars in 1322 bytes} [Thu Jun 30 12:19:06 2022] GET /identity/v3/role_assignments?user.id=bfba4b29b3e04e04b163f6e8cff16484&scope.domain.id=default&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:06.777214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 27/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu Jun 30 12:19:06 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:19:08.793503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a02d5-59f8-4493-8c60-ec3c717eba40 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:08.794336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a02d5-59f8-4493-8c60-ec3c717eba40 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:08.794336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a02d5-59f8-4493-8c60-ec3c717eba40 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:08.796592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 31/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:08.802713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:08.803101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:08.803279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:08.821533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:08.822754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:08.895794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 28/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:08.920530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a02d5-59f8-4493-8c60-ec3c717eba40 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:08.973449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:09.041229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a02d5-59f8-4493-8c60-ec3c717eba40 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:09.084333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0146c0e-cbe8-41d5-9687-18bdfe8caf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:09.085096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0146c0e-cbe8-41d5-9687-18bdfe8caf13 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:09.085382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0146c0e-cbe8-41d5-9687-18bdfe8caf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:09.085659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0146c0e-cbe8-41d5-9687-18bdfe8caf13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:09.086944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0146c0e-cbe8-41d5-9687-18bdfe8caf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:09.088416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0146c0e-cbe8-41d5-9687-18bdfe8caf13 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:09.090084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 32/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:09.096199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:09.112474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:09.206115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa2dc0-ed1f-47ae-8e57-b24c197724f0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:09.249775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7c18f-fe10-410f-82c7-a5ddbe3be992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:09.250542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7c18f-fe10-410f-82c7-a5ddbe3be992 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:09.250826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7c18f-fe10-410f-82c7-a5ddbe3be992 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:09.251113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7c18f-fe10-410f-82c7-a5ddbe3be992 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:09.251836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7c18f-fe10-410f-82c7-a5ddbe3be992 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:09.253373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7c18f-fe10-410f-82c7-a5ddbe3be992 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:09.284719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 29/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:09 2022] POST /identity/v3/users => generated 275 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:12.497339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11289cea-deb9-4484-a2f7-007bc9a8c51b None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.498392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11289cea-deb9-4484-a2f7-007bc9a8c51b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.498875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11289cea-deb9-4484-a2f7-007bc9a8c51b None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.500798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 33/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:12.514245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209216-5057-43b2-8912-24ecb840a2ac None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.515117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209216-5057-43b2-8912-24ecb840a2ac None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.515616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209216-5057-43b2-8912-24ecb840a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.541653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-fb209216-5057-43b2-8912-24ecb840a2ac None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:12.543264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fb209216-5057-43b2-8912-24ecb840a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:12.578585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e0851-f4c7-4a8a-a5f4-3ab3f5569be5 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.579211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e0851-f4c7-4a8a-a5f4-3ab3f5569be5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.579711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e0851-f4c7-4a8a-a5f4-3ab3f5569be5 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.581860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 34/63] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:12.587826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.588374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.588884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.612717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:12.614393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:12.621020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:12.626249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 30/64] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:12.627478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985d6a2-e8a6-47f7-98cb-6a048ea93f0a None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.627769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985d6a2-e8a6-47f7-98cb-6a048ea93f0a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.628047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985d6a2-e8a6-47f7-98cb-6a048ea93f0a None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.629496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 31/65] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:12.637968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.638502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.639026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.663460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:12.664906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:12.705259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 35/66] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:12.706150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:12.748181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:12.761012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 32/67] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:12.761012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:12.840516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad1d26-eb0f-4e81-8b72-145e5e75d349 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:12.878183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:12.879257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.879717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.880179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.887741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74042-5e43-40a7-a168-f7b8b08c70d4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:12.892209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:12.900705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:12.901666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:12.902432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:12.904952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 36/68] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:12.906019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:12.929599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:12.939077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:12.940168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:12.940459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:12.940773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:12.954778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:12.963751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:12.964611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:12.965394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:12.967813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 33/69] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:12.968862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.017120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.020762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb366c-ccc1-4941-9122-e2256919687d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.055658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d1a63-d346-4414-a96d-a52e01b7ffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.056806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1a63-d346-4414-a96d-a52e01b7ffed None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.057101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1a63-d346-4414-a96d-a52e01b7ffed None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.057378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1a63-d346-4414-a96d-a52e01b7ffed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.068851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d1a63-d346-4414-a96d-a52e01b7ffed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-745d1a63-d346-4414-a96d-a52e01b7ffed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:13.076554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:13.077610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:13.078591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 37/70] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:13.079423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d20ae-ff63-4d10-9d37-cdd60ff6fee0 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.079423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d20ae-ff63-4d10-9d37-cdd60ff6fee0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.079601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d20ae-ff63-4d10-9d37-cdd60ff6fee0 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.080908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 38/71] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:13.081755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d20ae-ff63-4d10-9d37-cdd60ff6fee0 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.084540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbb131-5379-45e8-90fb-8946331a4fd1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.120712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.121409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.121698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.121978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.122742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:13.123053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.124473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dd114-d334-4c6e-a523-08b9efe50707 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.137342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 34/72] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:13.137342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.137342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.137342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.150608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:13.151829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:13.201610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d20ae-ff63-4d10-9d37-cdd60ff6fee0 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.213125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.242297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 35/73] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:13.242297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.242557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.242557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.242557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.242557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.243120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:13.243120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.243120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.250498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 39/74] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:13.255284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.276176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.344003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1da4c-fc3f-4fe1-b83f-026ca029676a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.361960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725476f9-2d61-4bd0-ace1-37d0f8865e38 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.376617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.377321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.377605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.377888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.378648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] RBAC: Adding query filter params (name=member) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:13.378959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.380497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b02888-36f6-4f53-ac05-9fe506893658 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.388566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 36/75] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:13.394001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d2ce-9717-4358-821f-3de0248acc59 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.394297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d2ce-9717-4358-821f-3de0248acc59 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.394570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d2ce-9717-4358-821f-3de0248acc59 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.395855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 37/76] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:13.396549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d2ce-9717-4358-821f-3de0248acc59 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.396945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.397913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.398208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.398490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.412987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.414477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.416215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 40/77] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:13.420453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.446061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.501103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.501975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d2ce-9717-4358-821f-3de0248acc59 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.523867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1a0f6-7bd2-4ec2-ad12-f011412deb85 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.535074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.535913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.536197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.536476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.537730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.539198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.540890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 38/78] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:13.541657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.560279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f28335-8a74-42dd-a70b-30c5480845e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.561140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f28335-8a74-42dd-a70b-30c5480845e5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.561420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f28335-8a74-42dd-a70b-30c5480845e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.561690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f28335-8a74-42dd-a70b-30c5480845e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.572718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f28335-8a74-42dd-a70b-30c5480845e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f8f28335-8a74-42dd-a70b-30c5480845e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:13.580818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:13.581760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:13.582526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 41/79] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:13.585057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.585357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.585627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.602485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:13.603704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:13.642351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d093b5-1d23-4735-80fd-17a4a21537b1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.654884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.673295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 42/80] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:13.684785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.684907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.684907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.684907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.684907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.685200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.686577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.689005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 39/81] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:13.689686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.746399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.780046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536230d-3963-4dd2-b896-c9a876436738 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.786421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3f7ab-39c6-4c9e-b987-cfb30e1db38c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:13.815731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.816402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.816715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.816980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.818893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:13.819554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:13.819839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:13.820115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:13.869573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.883772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.885309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.889465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:13.890997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:13.953979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:13.987990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 43/82] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:13.988867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:13.993508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 40/83] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:13.994459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.072007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.088453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe9dc-0ffa-45eb-9b53-cab5ca54f75d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.091775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef161-bcbf-4499-a36d-5c4cdbb95f97 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.122347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.123028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.123303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.123575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.124320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:14.124648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.125438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.126581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.126871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.127150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.127613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.135930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 44/84] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.136749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.174576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.198119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.199632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.245134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8937dc-30ab-455d-a076-d0fc5404b468 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.276536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.277806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.278616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.278903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.279192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.280384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.281906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.283657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 45/85] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.284432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.301305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 41/86] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:14.302133 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.388312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.392416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9576e-ee35-48da-88ce-71bb27f4071c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.403205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caca24-9b13-49d9-819f-e63ea1d00c23 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.425697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.426583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.426866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.427145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.428335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.429834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.431549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 46/87] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.432311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.441618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.442500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.442784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.443070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.444644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.446128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.447995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 42/88] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.448591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.499346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.537632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552629-f4c2-4d15-8346-6b238d10d6c1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.551355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cfe96-f45f-439f-931e-dab8e43a81bf None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.573699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.574599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.574887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.575166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.576453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.577974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.579703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 47/89] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.580470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.586504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.587192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.587477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.587779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.588506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.590004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.611478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 43/90] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:14 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:14.612318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.617585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.696087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a520a4e-3c0a-431c-b999-e805f6bad6de None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.734699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.735670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.735955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.736233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.737567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.739063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.739333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189e55-f10a-4057-92d5-2c99fb30f515 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.740916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 48/91] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.741720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.746212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.784100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.784819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.785103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.785541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.786407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] RBAC: Adding query filter params (role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=d703bbb25adb40b5a2c4ac09ee36e0db) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:14.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.788214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.796585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 44/92] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/role_assignments?user.id=d703bbb25adb40b5a2c4ac09ee36e0db&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.797827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:14.882410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4eeeb-ec6c-4960-80e8-31a01b1b6ee6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.884793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:14.940837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:14.941889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:14.942363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:14.942840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:14.944154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=bfba4b29b3e04e04b163f6e8cff16484) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:14.944589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:14.946383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:14.951297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e8b5-195b-425d-99c6-8cbdb6f10ce6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:14.955627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 49/93] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/role_assignments?user.id=bfba4b29b3e04e04b163f6e8cff16484&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:14.956801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:15.000127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:15.001194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:15.001571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:15.001828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:15.069060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:15.111197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91261d-dbf6-4bc9-8e93-1abc2bdc12c4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:15.115626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:15.117202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:15.185868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:15.186548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:15.186830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:15.187103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:15.188109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=d703bbb25adb40b5a2c4ac09ee36e0db) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:15.188488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:15.190844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:15.201610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 50/94] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/role_assignments?user.id=d703bbb25adb40b5a2c4ac09ee36e0db&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:15.312874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 45/95] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:15.319091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:15.370079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:15.469436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911659f4-808d-4c0b-99f2-733049a7d76a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:15.511807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7437a8-9956-4dbf-b727-bd28db6579bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:15.512773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7437a8-9956-4dbf-b727-bd28db6579bc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:15.513056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7437a8-9956-4dbf-b727-bd28db6579bc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:15.513328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7437a8-9956-4dbf-b727-bd28db6579bc None admin] PATH_INFO: `/v3/projects/c721d943e62f442bb998096c9db0ee69` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:15.517479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7437a8-9956-4dbf-b727-bd28db6579bc None admin] RBAC: Authorizing `identity:get_project(project_id=c721d943e62f442bb998096c9db0ee69)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:15.519020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7437a8-9956-4dbf-b727-bd28db6579bc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:15.521055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 51/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:15 2022] GET /identity/v3/projects/c721d943e62f442bb998096c9db0ee69 => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:15.527329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:15.621939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:15.665340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e089d-8fb2-438d-9818-db02325645dd None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:15.701267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:15.701950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:15.702245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:15.702513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:15.703365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=c721d943e62f442bb998096c9db0ee69, user.id=d703bbb25adb40b5a2c4ac09ee36e0db) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:15.703752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:15.705260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1ee75-510f-4afb-a4fb-e69d907d0b74 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:15.711296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 46/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:15 2022] GET /identity/v3/role_assignments?user.id=d703bbb25adb40b5a2c4ac09ee36e0db&scope.project.id=c721d943e62f442bb998096c9db0ee69&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:17.638649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbff1cb-aba6-4747-aed1-f6a6f61ab2b9 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:17.639201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbff1cb-aba6-4747-aed1-f6a6f61ab2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:17.639201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbff1cb-aba6-4747-aed1-f6a6f61ab2b9 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:17.640740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 52/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:17.646614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:17.647409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:17.647832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:17.669073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:17.670512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:17.760310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 47/99] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:17.793262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:17.793681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1f85-8c8f-44b3-9db0-fcd7de461c1d None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:17.794195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1f85-8c8f-44b3-9db0-fcd7de461c1d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:17.794666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1f85-8c8f-44b3-9db0-fcd7de461c1d None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:17.796250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 53/100] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:17.802154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6aae6-c05d-4f6c-985f-c39aee8016b1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:17.802698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6aae6-c05d-4f6c-985f-c39aee8016b1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:17.803169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6aae6-c05d-4f6c-985f-c39aee8016b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:17.821361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cac6aae6-c05d-4f6c-985f-c39aee8016b1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:17.822912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cac6aae6-c05d-4f6c-985f-c39aee8016b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:17.901075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:17.945029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 54/101] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:17.946427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d9b1f-e162-4826-ae0b-e584083080f8 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:17.947514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d9b1f-e162-4826-ae0b-e584083080f8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:17.953099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d9b1f-e162-4826-ae0b-e584083080f8 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:17.955339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 55/102] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:17.962356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:17.962866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:17.964038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:17.968079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad8b7-da88-41b6-9667-cdb845233078 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.007355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:18.012038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:18.048987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.051893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.056939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.056939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.056939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.056939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:18.056939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:18.057941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 48/103] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:18.058495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.125428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 56/104] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:18.129698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.209739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df59d-6f4e-47be-a746-9b7d9a1020fc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.245235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.250150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627bdb7-2346-4f5c-979c-5e9509b1b932 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.252207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283f116-a8e5-4f45-8aa3-073257db0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.253188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f116-a8e5-4f45-8aa3-073257db0979 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.253471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f116-a8e5-4f45-8aa3-073257db0979 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.253742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f116-a8e5-4f45-8aa3-073257db0979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.264415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283f116-a8e5-4f45-8aa3-073257db0979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9283f116-a8e5-4f45-8aa3-073257db0979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:18.272028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:18.272904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:18.273662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 49/105] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:18.279408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350509ba-23b5-4891-a4c6-0702b8a137d5 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.279697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350509ba-23b5-4891-a4c6-0702b8a137d5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.279976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350509ba-23b5-4891-a4c6-0702b8a137d5 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.281546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 50/106] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:18.282118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350509ba-23b5-4891-a4c6-0702b8a137d5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.294731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d627c62-a0c4-41f1-8b3a-0bdaf3a21618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.295421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d627c62-a0c4-41f1-8b3a-0bdaf3a21618 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.295710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d627c62-a0c4-41f1-8b3a-0bdaf3a21618 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.295988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d627c62-a0c4-41f1-8b3a-0bdaf3a21618 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.296740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d627c62-a0c4-41f1-8b3a-0bdaf3a21618 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:18.298317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d627c62-a0c4-41f1-8b3a-0bdaf3a21618 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:18.329845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 57/107] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:19:18 2022] POST /identity/v3/projects => generated 302 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:18.335219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e7fbf-0d7a-40d9-b7a8-11a2201553ca None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e7fbf-0d7a-40d9-b7a8-11a2201553ca None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.336212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e7fbf-0d7a-40d9-b7a8-11a2201553ca None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.342818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 58/108] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:18.342818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e7fbf-0d7a-40d9-b7a8-11a2201553ca None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.358895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.414183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350509ba-23b5-4891-a4c6-0702b8a137d5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.462880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509a7d0-47a3-454f-a437-4012417c0d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.463999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509a7d0-47a3-454f-a437-4012417c0d13 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.464287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509a7d0-47a3-454f-a437-4012417c0d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.464566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509a7d0-47a3-454f-a437-4012417c0d13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.480178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509a7d0-47a3-454f-a437-4012417c0d13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:18.484869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e7fbf-0d7a-40d9-b7a8-11a2201553ca None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a509a7d0-47a3-454f-a437-4012417c0d13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:18.490768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:18.491669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:18.492703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 51/109] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:18.493848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.494302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.494596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.516046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:18.517473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:18.532335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.533359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.533854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.534283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.535059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:18.535631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:18.538406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1900e-ef19-44e3-8832-c3307a8229d4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:18.549529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 59/110] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:18.551184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.551184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.551458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.573356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:18.575033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:18.585179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.613639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 52/111] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:18.613639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.676658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 60/112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:18.677595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.686438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.776288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9df71-3572-4df8-958c-c59c6a2300bf None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.801481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcddc6-d8c7-4d06-85dc-08346156a68f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.814830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.815697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.816048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.816429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.817292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] RBAC: Adding query filter params (name=member) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:18.817679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:18.819557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:18.829226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 53/113] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:18.830310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.848691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:18.849711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:18.849998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:18.850534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:18.852136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:18.853661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:18.856284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 61/114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:18.856908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:18.886306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.949850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:18.960273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4f251-173a-4b1c-9bd2-804efe327521 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.986108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdd48a-efa4-4e26-bb72-114ba62a554b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:18.999925 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.001406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.001728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.001988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.013425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:19.021528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:19.022445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:19.023237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 54/115] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:19.024496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.028866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.029726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.030016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.030294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.042146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:19.050610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:19.051555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 62/116] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:19.057374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.136506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450008-0342-479e-b814-ace5d7f60440 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.138197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:19.169455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ced2-9d44-43f7-9f64-8eb1f920fd18 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.178779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8340f-ecc3-453d-a927-4716b8fc44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.179646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8340f-ecc3-453d-a927-4716b8fc44cd None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.179963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8340f-ecc3-453d-a927-4716b8fc44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.180241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8340f-ecc3-453d-a927-4716b8fc44cd None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.181569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8340f-ecc3-453d-a927-4716b8fc44cd None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.183060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8340f-ecc3-453d-a927-4716b8fc44cd None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.184834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 55/117] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.193139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62ee4a-52e3-4160-ba15-62a08cd1f35f None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.193139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62ee4a-52e3-4160-ba15-62a08cd1f35f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.193139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62ee4a-52e3-4160-ba15-62a08cd1f35f None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.193139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 56/118] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:19.193139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62ee4a-52e3-4160-ba15-62a08cd1f35f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.210482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:19.217149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.218034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.218321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.218601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.220146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.221674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.228430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 63/119] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.228430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.319725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62ee4a-52e3-4160-ba15-62a08cd1f35f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.352110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105a345-4327-4493-ad65-e6ac2034e1e0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.358285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.359022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.359308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.359585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.360502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:19.360844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.362265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.371507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 57/120] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.372384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.400762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976844db-649a-4744-b84c-9136042672cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.401694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976844db-649a-4744-b84c-9136042672cc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.401978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976844db-649a-4744-b84c-9136042672cc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.402252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976844db-649a-4744-b84c-9136042672cc None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.403008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976844db-649a-4744-b84c-9136042672cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:19.403318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976844db-649a-4744-b84c-9136042672cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.404776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976844db-649a-4744-b84c-9136042672cc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.407332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:19.420568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 64/121] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.420568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.420568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.420568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.438364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:19.439657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:19.488017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f995254-7804-43e7-87cd-077abb7a357d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.500504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:19.520666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 65/122] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:19.520666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.541697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.542581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.542865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.543139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.544695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.545866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.547905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 58/123] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.548803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.631728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2463-4f24-4baa-b00d-33ea25aca2a0 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.669279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.670173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.670467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.670736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/bfba4b29b3e04e04b163f6e8cff16484/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.671290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:19.673007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbe7793149f98e33576b094aa8a2, user_id=bfba4b29b3e04e04b163f6e8cff16484, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.674484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.674604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a094a3-0b69-414c-8b2b-0f26a3f1c4e6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.700665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 66/124] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:19 2022] PUT /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/bfba4b29b3e04e04b163f6e8cff16484/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:19.700665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.732465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.733902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.734252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.735130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] PATH_INFO: `/v3/users/f3b64ee5fd1749afa35c9275f6ed6611` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.758819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] RBAC: Authorizing `identity:get_user(user_id=f3b64ee5fd1749afa35c9275f6ed6611)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.759728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:19.762063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.766258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 59/125] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/users/f3b64ee5fd1749afa35c9275f6ed6611 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.767639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.822586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493009a-a77f-417e-baad-4b455a8f10ec None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.863617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.864703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.865220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.865867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.867262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:19.868821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:19.870566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 67/126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:19.871410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:19.921856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b7da2-d5fb-47b1-b687-6b4e3a0e7b8b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.969944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:19.971109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:19.971638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:19.971921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:19.987117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:19.994420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2420584-e023-4a61-94de-7b92c28ecb62 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:19.997561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:19.998604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:19.999559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 60/127] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:20.000199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.023147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:20.036163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.037539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.037987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.038406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/d703bbb25adb40b5a2c4ac09ee36e0db/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.040771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbe7793149f98e33576b094aa8a2, user_id=d703bbb25adb40b5a2c4ac09ee36e0db, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.042700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.057087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:20.060682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 68/128] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:19 2022] PUT /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/d703bbb25adb40b5a2c4ac09ee36e0db/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:20.062025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.113017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e77e27-1621-4397-bcd6-47bbd8ee117f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.150979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.151265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.151545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.196948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967a79-3c5e-474c-a99a-61629b1fcfcd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.231457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.233151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.240555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.241981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.242478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.242954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.244564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.246378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.248486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 69/129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:20 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:20.249591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.257707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:20.325066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:20.344051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 61/130] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:20.345440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.377721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02476d-fed5-48e4-9d68-64da0f2f3da1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.416278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.417371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.417885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.418351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.419293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:20.419888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.421593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.432804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 70/131] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:20.434122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.452944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e51e1b-af64-410d-86c7-e7da1dec4d34 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.471451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:20.488766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.489690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.490038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.490254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/d703bbb25adb40b5a2c4ac09ee36e0db/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.492750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbe7793149f98e33576b094aa8a2, user_id=d703bbb25adb40b5a2c4ac09ee36e0db, role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.494082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.517475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 62/132] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:20 2022] PUT /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/d703bbb25adb40b5a2c4ac09ee36e0db/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:20.517947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.555636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5396b3-f159-4905-b6a4-f690392f4f58 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.566965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:20.618420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.619707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.620179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.620669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.622138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.623873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.626661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 71/133] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:20 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:20.637249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.788043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe549e8-4010-4b07-952a-7a99f8fa625f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.810854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a76b-cf5f-48f1-8443-866dbba309e7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:20.837156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.837840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.838125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.838403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.839251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] RBAC: Adding query filter params (role.id=060c3431b7d24ac98e46d6a8f9e4c556, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=f3b64ee5fd1749afa35c9275f6ed6611) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:20.839564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.841087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088822b2-662a-43df-8da8-7720a9ba185c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.855523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 72/134] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:20 2022] GET /identity/v3/role_assignments?user.id=f3b64ee5fd1749afa35c9275f6ed6611&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=060c3431b7d24ac98e46d6a8f9e4c556 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:20.873688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:20.874607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.874880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.875165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.881010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3134f76-3656-4c49-b4b2-e99823e81438 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.881333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3134f76-3656-4c49-b4b2-e99823e81438 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.881579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3134f76-3656-4c49-b4b2-e99823e81438 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.883039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 73/135] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:20.883802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:20.885369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:20.887361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 63/136] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:20 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:20.889073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:20.889363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:20.889642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:20.901319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:20.921877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:20.923304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:21.021022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:21.025016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:21.075993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 74/137] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:21.083785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81805143-ba18-421d-9c0a-f38a1f37d07d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:21.110808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:21.121917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:21.122862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:21.123146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:21.123419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] PATH_INFO: `/v3/projects/c721d943e62f442bb998096c9db0ee69` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:21.126121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] RBAC: Authorizing `identity:get_project(project_id=c721d943e62f442bb998096c9db0ee69)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:21.129517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:21.129517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 64/138] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:20 2022] GET /identity/v3/projects/c721d943e62f442bb998096c9db0ee69 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:21.134635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:21.281399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a293c-0718-4411-83b9-d7a7ce80805f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:21.306750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53036ca0-6a93-45b8-becb-631e31b4a58e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:21.322919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:21.332969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:21.333987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:21.334269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:21.334576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:21.336505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:21.338798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:21.341289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 75/139] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:21.347891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:21.351257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1823c-0ef6-4254-81a5-734d2f5ccd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:21.352345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1823c-0ef6-4254-81a5-734d2f5ccd5a None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:21.352771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1823c-0ef6-4254-81a5-734d2f5ccd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:21.353146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1823c-0ef6-4254-81a5-734d2f5ccd5a None admin] PATH_INFO: `/v3/projects/c721d943e62f442bb998096c9db0ee69/users/d703bbb25adb40b5a2c4ac09ee36e0db/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:21.355503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1823c-0ef6-4254-81a5-734d2f5ccd5a None admin] RBAC: Authorizing `identity:create_grant(project_id=c721d943e62f442bb998096c9db0ee69, user_id=d703bbb25adb40b5a2c4ac09ee36e0db, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:21.357132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1823c-0ef6-4254-81a5-734d2f5ccd5a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:21.385858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 65/140] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:21 2022] PUT /identity/v3/projects/c721d943e62f442bb998096c9db0ee69/users/d703bbb25adb40b5a2c4ac09ee36e0db/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:21.407094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:21.484421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8cfa7-00d7-481f-b20d-586189ee44ca None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:21.521832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766fb79-feff-44ae-822e-3062d0896a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:21.522529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766fb79-feff-44ae-822e-3062d0896a90 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:21.522815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766fb79-feff-44ae-822e-3062d0896a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:21.523094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766fb79-feff-44ae-822e-3062d0896a90 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:21.523890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766fb79-feff-44ae-822e-3062d0896a90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:21.525439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766fb79-feff-44ae-822e-3062d0896a90 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:21.567203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 76/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:21 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:19:22.502175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2cf8-a363-4302-a0cd-1f74c47fea03 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:22.502724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2cf8-a363-4302-a0cd-1f74c47fea03 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:22.502724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2cf8-a363-4302-a0cd-1f74c47fea03 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:22.504464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 66/142] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:22.509993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae791896-2e99-4e6e-9746-329175f155e7 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:22.510437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae791896-2e99-4e6e-9746-329175f155e7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:22.510522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae791896-2e99-4e6e-9746-329175f155e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:22.533580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ae791896-2e99-4e6e-9746-329175f155e7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:22.534842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ae791896-2e99-4e6e-9746-329175f155e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:22.617679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 77/143] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:22.648028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2cf8-a363-4302-a0cd-1f74c47fea03 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:22.762112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3045-7f5b-4073-8360-e34dc5949db9 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:22.762416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3045-7f5b-4073-8360-e34dc5949db9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:22.762594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3045-7f5b-4073-8360-e34dc5949db9 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:22.764119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 78/144] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:22.769595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:22.769878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:22.770157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:22.775779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2cf8-a363-4302-a0cd-1f74c47fea03 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:22.805900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:22.807127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:22.829289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:22.830618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:22.831100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:22.831565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:22.842162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:22.852659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:22.860649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:22.861585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:22.862928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 67/145] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:22.867970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:22.935245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 79/146] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:22.953074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:23.019677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:23.067540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8263f-c36f-491e-be1f-afc0a8702d52 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:23.129094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:23.129782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.130065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.130344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.131109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:23.131421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:23.134285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:23.149091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 68/147] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:23.154634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:23.204655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d20321-ef4f-4b78-ba04-b293c4c9cf24 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:23.268385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:23.269227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.269508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.269827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.284383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:23.293426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:23.294324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:23.295540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 80/148] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:23.300972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:23.322582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:23.328547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:23.337181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eed28-019e-4b95-8f73-bbe35af15720 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:23.383479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf084ed3-1e58-4f37-b77e-c43c7fee773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:23.384392 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf084ed3-1e58-4f37-b77e-c43c7fee773a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.384716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf084ed3-1e58-4f37-b77e-c43c7fee773a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.384999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf084ed3-1e58-4f37-b77e-c43c7fee773a None admin] PATH_INFO: `/v3/users/bfba4b29b3e04e04b163f6e8cff16484` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.386237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf084ed3-1e58-4f37-b77e-c43c7fee773a None admin] RBAC: Authorizing `identity:get_user(user_id=bfba4b29b3e04e04b163f6e8cff16484)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:23.387725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf084ed3-1e58-4f37-b77e-c43c7fee773a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:23.389734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 69/149] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:23 2022] GET /identity/v3/users/bfba4b29b3e04e04b163f6e8cff16484 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:23.391061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fad73-2240-472d-89e7-58306d693aef None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.391426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fad73-2240-472d-89e7-58306d693aef None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.391720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fad73-2240-472d-89e7-58306d693aef None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.393005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 70/150] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:23.398417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.398700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.399199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.418689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:23.419897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:23.434128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44ba69-b485-4b38-8746-670e51e5b232 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:23.495302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:23.495988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.496272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.496547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.497339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:23.497643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:23.500345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:23.516235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 81/151] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:23.517161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:23.525102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 71/152] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:23.525940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:23.694769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:23.708875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9efa1-8a36-4403-a9c5-51697f1601bf None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:23.709198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:23.748101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84361-c3f4-4ea9-abb7-d3cf9124390f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:23.789711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:23.790298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.790580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.790855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.813443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:23.814367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:23.814652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:23.814932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:23.816446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:23.817947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:23.820034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 72/153] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:23 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:23.825746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:23.883680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:23.885201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:23.979432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fdeac-c4a3-4729-ac93-90a68781e0d0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.019543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.020515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.020821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.021096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.033038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:24.042693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:24.043560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 73/154] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:24.045143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.074209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.095407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.151598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 82/155] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:24.152482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.170848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83652f92-3d85-43ef-ae1f-1c18b4d70b5a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.209869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.210655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.210940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.211368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.307961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:24.309428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:24.346414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.394526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ab92b-88b3-4265-a8b0-c54b3e13fc7c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.455798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 74/156] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:24.461101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.483685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.484382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.484681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.484959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.485871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:24.486270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:24.487698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03a034-d6a8-47a5-86b8-8f604f3ce1e8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:24.507118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 83/157] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:24.508560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14059e-6731-4ac9-89e3-2f6829185efa None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.508885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14059e-6731-4ac9-89e3-2f6829185efa None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.509161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14059e-6731-4ac9-89e3-2f6829185efa None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.510811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 84/158] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:24.511352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14059e-6731-4ac9-89e3-2f6829185efa None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.574228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b91d1-8ff6-474a-b059-33ff7d5b0883 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.600420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.612372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.614196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18113f75-2472-4f1a-b105-907914c81834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.615092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18113f75-2472-4f1a-b105-907914c81834 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.615373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18113f75-2472-4f1a-b105-907914c81834 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.615645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18113f75-2472-4f1a-b105-907914c81834 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.616958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18113f75-2472-4f1a-b105-907914c81834 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:24.618842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18113f75-2472-4f1a-b105-907914c81834 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:24.621421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 75/159] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:24.622606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.622889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.623476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.647428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:24.649387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:24.710089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14059e-6731-4ac9-89e3-2f6829185efa None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.728788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 76/160] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:24.729650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.746402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.747432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.747810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.748086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.749323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:24.751413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:24.753445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 85/161] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:24.754018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.858904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d5072-dd70-4707-af68-0003708627f0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.865981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6c94-1182-4ecc-97da-56f159047253 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:24.873543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.883221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:24.913783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.914676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.914953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.915224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.916573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:24.919434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:24.921358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 77/162] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:24.929128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:24.931006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:24.931685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:24.931967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:24.932240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:24.933110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=bfba4b29b3e04e04b163f6e8cff16484) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:24.933422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:24.936644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:24.955934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 86/163] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/role_assignments?user.id=bfba4b29b3e04e04b163f6e8cff16484&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:24.956447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.119568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bf37d-efc0-44ad-802f-a3931bf5cafa None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.147970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488ce65-e900-4ed4-a8dc-c16deff5f937 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.190045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.190742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.191023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.191295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.192163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=d703bbb25adb40b5a2c4ac09ee36e0db) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:25.192476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:25.194393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:25.203349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 78/164] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/role_assignments?user.id=d703bbb25adb40b5a2c4ac09ee36e0db&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:25.205935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.221124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.221957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.222246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.222512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.241816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:25.249769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:25.250682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:25.251498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:25.258463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 87/165] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:25.266626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.312283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:25.345032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:25.386646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548bfe8-8ec8-4695-8d52-f1e3cd499750 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.406723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9721da-9953-4ada-a09d-37fc28b33485 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.432000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.436741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.437128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.437408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.447890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.448660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.448957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.449233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.450001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:25.450308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:25.451966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:25.464809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 88/166] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:25.471466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.532568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:25.534156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:25.593028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:25.607206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005a87d-ad50-4510-a64d-37d6c25e67f3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.615894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:25.664441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.665312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.665542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.665812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] PATH_INFO: `/v3/users/f3b64ee5fd1749afa35c9275f6ed6611` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.668208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b64ee5fd1749afa35c9275f6ed6611)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:25.670376 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:25.671816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 89/167] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:25 2022] GET /identity/v3/users/f3b64ee5fd1749afa35c9275f6ed6611 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:25.679052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.706017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 79/168] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:25.725754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.864971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05eff9c-c114-42e0-8707-7adb4c8edcae None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.867596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86b8f5-73e2-4d78-8fc6-100e8908c5f9 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:25.912347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b962a-8f6d-4b88-8c04-69668574b57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.913251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b962a-8f6d-4b88-8c04-69668574b57d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.913532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b962a-8f6d-4b88-8c04-69668574b57d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.913797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b962a-8f6d-4b88-8c04-69668574b57d None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.915023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b962a-8f6d-4b88-8c04-69668574b57d None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:25.915967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ea471-a88c-446b-bee0-8720481538fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:25.916429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b962a-8f6d-4b88-8c04-69668574b57d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:25.917313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ea471-a88c-446b-bee0-8720481538fd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.917849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ea471-a88c-446b-bee0-8720481538fd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.918311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ea471-a88c-446b-bee0-8720481538fd None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.918522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 80/169] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:25 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:25.924121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b31a0-42b4-4708-986f-27d6de04a311 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.924121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b31a0-42b4-4708-986f-27d6de04a311 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.924121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b31a0-42b4-4708-986f-27d6de04a311 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.924121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 81/170] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:25.925460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ea471-a88c-446b-bee0-8720481538fd None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:25.926603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b31a0-42b4-4708-986f-27d6de04a311 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:25.927433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ea471-a88c-446b-bee0-8720481538fd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:25.929765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 90/171] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:25 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:25.931315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:25.931796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:25.932294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:25.959494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:25.960724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:25.981232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.006892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.070753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 91/172] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:26.077481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.096494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b31a0-42b4-4708-986f-27d6de04a311 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:26.143647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:26.144972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.145844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.146430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.147551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] RBAC: Adding query filter params (role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=d703bbb25adb40b5a2c4ac09ee36e0db) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:26.148262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:26.150313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1b7f1-b007-46a9-91eb-afdf541e8b98 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:26.159110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 82/173] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:25 2022] GET /identity/v3/role_assignments?user.id=d703bbb25adb40b5a2c4ac09ee36e0db&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:26.160924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6977f-8e05-43b3-9b31-2e64cfad3540 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.161592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6977f-8e05-43b3-9b31-2e64cfad3540 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.162438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6977f-8e05-43b3-9b31-2e64cfad3540 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.164546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 83/174] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:26.177739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6977f-8e05-43b3-9b31-2e64cfad3540 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.210968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a24c-21d7-405e-a13f-570a10974d5d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:26.257814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235eeb44-5afa-47b0-8489-b8f7a88bdcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:26.258729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eeb44-5afa-47b0-8489-b8f7a88bdcc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.259008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eeb44-5afa-47b0-8489-b8f7a88bdcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.259282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eeb44-5afa-47b0-8489-b8f7a88bdcc4 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/f3b64ee5fd1749afa35c9275f6ed6611/roles/060c3431b7d24ac98e46d6a8f9e4c556` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.262556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eeb44-5afa-47b0-8489-b8f7a88bdcc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbe7793149f98e33576b094aa8a2, user_id=f3b64ee5fd1749afa35c9275f6ed6611, role_id=060c3431b7d24ac98e46d6a8f9e4c556)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:26.265570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235eeb44-5afa-47b0-8489-b8f7a88bdcc4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:26.285065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 92/175] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:26 2022] PUT /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/users/f3b64ee5fd1749afa35c9275f6ed6611/roles/060c3431b7d24ac98e46d6a8f9e4c556 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:26.287690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.288259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.288757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.309035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.312728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:26.315434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:26.340043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.346211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6977f-8e05-43b3-9b31-2e64cfad3540 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:26.416420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 93/176] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:26.420187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:26.424747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.424747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.424747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.446329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:26.457607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:26.458604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:26.459393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 84/177] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:26.465262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.473798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.595691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bccc-88f3-4aff-8ba2-d15a6684eef9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:26.636720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:26.637403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.638175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.638449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.639361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:26.639670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:26.641167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:26.651662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 94/178] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:26.659160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.684050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.712964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095586-7570-4e74-9f24-886fe459e6e8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:26.762187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2b8f0-9181-4564-ada0-3d712cbf5524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:26.764240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2b8f0-9181-4564-ada0-3d712cbf5524 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.764521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2b8f0-9181-4564-ada0-3d712cbf5524 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.764819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2b8f0-9181-4564-ada0-3d712cbf5524 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.785143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2b8f0-9181-4564-ada0-3d712cbf5524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-52b2b8f0-9181-4564-ada0-3d712cbf5524 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:26.793001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:26.793866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:26.794641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:26.799864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 85/179] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:26.799864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db7fb1-463e-44f6-ad16-d23d50b911fa None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.799864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db7fb1-463e-44f6-ad16-d23d50b911fa None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.799864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db7fb1-463e-44f6-ad16-d23d50b911fa None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.799864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 86/180] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:26.804761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72379b99-0760-4f80-b7b5-1a579643a26d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:26.806322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.806614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.806878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.814503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.826170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:26.827476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:26.852989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:26.854508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:26.855171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:26.856033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] PATH_INFO: `/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:26.857757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] RBAC: Authorizing `identity:get_user(user_id=d703bbb25adb40b5a2c4ac09ee36e0db)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:26.860697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:26.862761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 95/181] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:26 2022] GET /identity/v3/users/d703bbb25adb40b5a2c4ac09ee36e0db => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:26.864788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.904147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 87/182] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:26.905045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:26.970123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:26.995799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881bcb0-2911-445f-951b-02cbfd8c3369 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.013408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77090446-0a1f-46c5-99f3-feeed9fcdb34 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.031325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.032025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.032527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.033144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.033899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:27.034204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.035594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf85c2-d063-46d2-939b-e6907bd644f5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.048729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 96/183] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:27.048729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8a0c-c3de-4db1-8c84-7683608d621d None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.048729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8a0c-c3de-4db1-8c84-7683608d621d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.048729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8a0c-c3de-4db1-8c84-7683608d621d None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.048729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 97/184] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:27.049118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8a0c-c3de-4db1-8c84-7683608d621d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.057409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd04789-fd3d-4125-8976-5653bbb8d3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.058780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd04789-fd3d-4125-8976-5653bbb8d3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.059269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd04789-fd3d-4125-8976-5653bbb8d3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.059737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd04789-fd3d-4125-8976-5653bbb8d3a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.063782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.136315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd04789-fd3d-4125-8976-5653bbb8d3a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.137891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd04789-fd3d-4125-8976-5653bbb8d3a0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.157342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8a0c-c3de-4db1-8c84-7683608d621d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.195986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.196800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.197097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.197368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.198551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.200017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.206373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 98/185] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:27.206373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.215784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.259556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 88/186] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:27.259556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.259556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.259556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.272880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:27.274090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:27.303187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.320937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ccf40-9c25-4ab4-bafb-8b1268fe8b8b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.360679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 89/187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:27.360679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.369797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.370706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.370989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.371582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] PATH_INFO: `/v3/users/dccdda8cf5894e76a4bf0315694830eb` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.387021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] RBAC: Authorizing `identity:get_user(user_id=dccdda8cf5894e76a4bf0315694830eb)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.388720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.390410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 99/188] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/users/dccdda8cf5894e76a4bf0315694830eb => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:27.395548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.464791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.466314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dd581-50e8-4685-b81d-82bce266809c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.499805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a12838-c0f0-458c-b06b-808d890f5290 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.503640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.504492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.504942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.505357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.506230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.507836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.531136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 90/189] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:27 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:27.532329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.545149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.546049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.546329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.546612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] PATH_INFO: `/v3/projects/c721d943e62f442bb998096c9db0ee69` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.548110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] RBAC: Authorizing `identity:get_project(project_id=c721d943e62f442bb998096c9db0ee69)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.549877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.556607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 100/190] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/projects/c721d943e62f442bb998096c9db0ee69 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:27.556607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.576675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.681535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05ffd3-0fd0-440e-b531-b64682c3b3ef None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.690398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ea12c-4f74-4e67-837e-508c5e5004eb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.721562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.722227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.722505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.722968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.726723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.732610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.734020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.734436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.734842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.748745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:27.758704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:27.759772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:27.760732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:27.762741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 91/191] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:27.763712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.812757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.812757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.878788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:27.900869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d798ee-c936-481c-9543-8c95c6a2f835 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:27.942721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 101/192] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:27.944110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.945689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:27.946566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:27.946988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:27.947401 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:27.948390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=c721d943e62f442bb998096c9db0ee69, user.id=d703bbb25adb40b5a2c4ac09ee36e0db) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:27.948940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:27.950584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:27.961704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 92/193] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/role_assignments?user.id=d703bbb25adb40b5a2c4ac09ee36e0db&scope.project.id=c721d943e62f442bb998096c9db0ee69&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:27.966516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:27.984676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:28.056328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466c714-ec83-4878-8529-36ba141dcaab None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:28.116040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90c802-b428-4b79-8a02-1593a0930c74 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:28.117844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:28.118638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.119022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.119362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.120804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:28.120804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:28.122044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:28.139764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 102/194] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:28.145653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:28.155816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:28.157219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.157678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.158121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.159603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:28.161299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:28.163236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 93/195] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:27 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:28.174048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:28.190826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:28.290196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0415-0b4f-4f62-8851-71b93b57b02d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:28.329044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:28.330085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.330572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.331043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.332089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=dccdda8cf5894e76a4bf0315694830eb) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:28.332648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:28.332751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27a8d8-15b3-4233-914b-e4ad1f3bdc21 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:28.334551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:28.341284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 94/196] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:28 2022] GET /identity/v3/role_assignments?user.id=dccdda8cf5894e76a4bf0315694830eb&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:28.346395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:28.394988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb162203-2a2d-43d7-b1a9-f0fc2425fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:28.395827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162203-2a2d-43d7-b1a9-f0fc2425fe1c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.396110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162203-2a2d-43d7-b1a9-f0fc2425fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.396386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162203-2a2d-43d7-b1a9-f0fc2425fe1c None admin] PATH_INFO: `/v3/users/dccdda8cf5894e76a4bf0315694830eb` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.401516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb162203-2a2d-43d7-b1a9-f0fc2425fe1c None admin] RBAC: Authorizing `identity:get_user(user_id=dccdda8cf5894e76a4bf0315694830eb)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:28.403051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb162203-2a2d-43d7-b1a9-f0fc2425fe1c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:28.404928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 103/197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:28 2022] GET /identity/v3/users/dccdda8cf5894e76a4bf0315694830eb => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:28.410585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:28.532155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:28.559320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35169-0d08-4d9f-ba3e-16ec31daa0d4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:28.595975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:28.597023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.597504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.597973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.669699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:28.671560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:28.689517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10027dbc-6370-4fc8-9a0f-06ab6de97201 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.689800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10027dbc-6370-4fc8-9a0f-06ab6de97201 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.690311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10027dbc-6370-4fc8-9a0f-06ab6de97201 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.691913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 104/198] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:28.697663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.698222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.698705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.716021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:28.717607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:28.759272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:28.792652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 105/199] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:28.806279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 95/200] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:28.811490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:28.825843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:28.910258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:28.917379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2749e61-3d93-4906-867d-1f823b014ca9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:28.958994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:28.960260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:28.960791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:28.961262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:28.962698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:28.964374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:28.966486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 106/201] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:28 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:28.978043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:28.993752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b406f3-65e9-4938-8707-f9fa3ee59c85 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:29.068473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:29.069693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:29.069972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:29.070247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:29.094105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:29.102477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:29.103400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:29.104302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 96/202] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:29.109615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:29.136540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5ba31-4323-472d-b4a8-707d48d0565b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:29.174750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:29.176447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:29.176949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:29.177051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:29.178236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:29.179452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] RBAC: Adding query filter params (role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=dccdda8cf5894e76a4bf0315694830eb) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:29.181369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:29.183820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:29.197488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 107/203] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:28 2022] GET /identity/v3/role_assignments?user.id=dccdda8cf5894e76a4bf0315694830eb&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:29.259168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf918232-8f5f-4fef-8e36-b629182a41a8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:29.303387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:29.304413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:29.304934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:29.305447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:29.306483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:29.307009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:29.308722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:29.319537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 97/204] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:29.325185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:29.368651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:29.465289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731144d-f175-4567-b906-9e510d60c08a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:29.501605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:29.502512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:29.502793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:29.503067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] PATH_INFO: `/v3/users/f3b64ee5fd1749afa35c9275f6ed6611` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:29.504298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b64ee5fd1749afa35c9275f6ed6611)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:29.506119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:29.508098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 108/205] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:29 2022] GET /identity/v3/users/f3b64ee5fd1749afa35c9275f6ed6611 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:29.513716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:29.623816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:29.632918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1985ae3-1ca4-43d3-959b-12c385e3c1e4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:29.669447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:29.670156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:29.670440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:29.670713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:29.738255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:29.739762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:29.836858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:29.841763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 98/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:29.850430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:29.988699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:29.995795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5126d-678b-4946-b95a-6e2d05a48318 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:30.033084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93de86-723c-4808-a8e9-6dca760b875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:30.033970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93de86-723c-4808-a8e9-6dca760b875f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.034250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93de86-723c-4808-a8e9-6dca760b875f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.034528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93de86-723c-4808-a8e9-6dca760b875f None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.035720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93de86-723c-4808-a8e9-6dca760b875f None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:30.037240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93de86-723c-4808-a8e9-6dca760b875f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:30.039262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 109/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:29 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:30.045123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:30.198222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2494-f5bb-4c59-ac6a-74f208ac0208 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:30.254303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:30.254983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.255265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.255540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.256403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] RBAC: Adding query filter params (role.id=060c3431b7d24ac98e46d6a8f9e4c556, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2, user.id=f3b64ee5fd1749afa35c9275f6ed6611) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:30.256740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:30.258195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b96dd0-11e1-46e2-8a0c-e44373c17430 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:30.265175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 99/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:30 2022] GET /identity/v3/role_assignments?user.id=f3b64ee5fd1749afa35c9275f6ed6611&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=060c3431b7d24ac98e46d6a8f9e4c556 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:30.534793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584c8b6-f74d-4ebc-9d8b-4d9cd85a67a7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.535348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584c8b6-f74d-4ebc-9d8b-4d9cd85a67a7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.535768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584c8b6-f74d-4ebc-9d8b-4d9cd85a67a7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.537614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 110/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:30.542533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.543082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.543565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.564394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:30.566026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:30.637774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:30.645215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61780a52-41d6-4a59-8d0d-1a813ae246fe None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.645813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61780a52-41d6-4a59-8d0d-1a813ae246fe None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.646322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61780a52-41d6-4a59-8d0d-1a813ae246fe None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.648452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 111/210] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:30.650443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 100/211] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:30.656013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.656538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.657107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.675911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:30.676427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:30.677975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:30.754295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 112/212] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:30.780358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:30.787718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb479d7-2881-4678-9407-279b384dc69a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:30.824587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:30.825651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.826081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.826515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.827891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:30.829606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:30.831408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 101/213] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:30.836952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:30.881759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:30.892796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1c66-b091-4a7b-b160-b55da3079184 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:30.902786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:30.928829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555f9d-1dfe-4c61-b249-aa1fc80ecbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:30.929613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555f9d-1dfe-4c61-b249-aa1fc80ecbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.929890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555f9d-1dfe-4c61-b249-aa1fc80ecbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.930333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555f9d-1dfe-4c61-b249-aa1fc80ecbd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.941237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555f9d-1dfe-4c61-b249-aa1fc80ecbd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:30.947856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887925e-50d4-4a0a-9e57-53536d441495 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ad555f9d-1dfe-4c61-b249-aa1fc80ecbd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:30.949317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:30.950409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 113/214] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:30.953431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831747-7ce3-4133-8f1d-ece78866e2a7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.954207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831747-7ce3-4133-8f1d-ece78866e2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.954207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831747-7ce3-4133-8f1d-ece78866e2a7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.954207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 114/215] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:30.958621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c831747-7ce3-4133-8f1d-ece78866e2a7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:30.994362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf9bfc-6e74-4bee-abcf-a62d7523f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:30.995128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf9bfc-6e74-4bee-abcf-a62d7523f7de None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:30.995477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf9bfc-6e74-4bee-abcf-a62d7523f7de None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:30.995813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf9bfc-6e74-4bee-abcf-a62d7523f7de None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:30.996605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf9bfc-6e74-4bee-abcf-a62d7523f7de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:30.998088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf9bfc-6e74-4bee-abcf-a62d7523f7de None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.020562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 102/216] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:30 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:31.021920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.022210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.022809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.046361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:31.047607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:31.076422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c831747-7ce3-4133-8f1d-ece78866e2a7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.120384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.121224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.121549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.121826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.122754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:31.123125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:31.124795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.135399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 103/217] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:31.141289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 115/218] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:31.142641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:31.162274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:31.188216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:31.217077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:31.281285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b2092-e36c-42d0-9f7f-8a5508d73bc3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.289201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540d1ad-6ffd-48de-8ac5-ffd04ffd04f2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.336777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.337902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.337902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.338069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] PATH_INFO: `/v3/users/25145350d12d4bfd8cdf5054e8832835` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.342043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.343176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.343678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.344159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.357594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] RBAC: Authorizing `identity:get_user(user_id=25145350d12d4bfd8cdf5054e8832835)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:31.359360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.359905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:31.361483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 104/219] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/users/25145350d12d4bfd8cdf5054e8832835 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:31.367896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:31.369135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:31.370031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:31.370814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:31.371871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 116/220] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:31.380190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:31.479613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:31.493655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c9573-940a-4435-ae8a-cbbb0f9d8921 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.496948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:31.527793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e269-2f00-4bdf-8c00-ee22dfd8794a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.560815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b8901-0bcf-4de0-9455-b7945903956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.561484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8901-0bcf-4de0-9455-b7945903956c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.561767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8901-0bcf-4de0-9455-b7945903956c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.562036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8901-0bcf-4de0-9455-b7945903956c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.613471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.614085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.614367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.614644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.615409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:31.615726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:31.617206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.632921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8901-0bcf-4de0-9455-b7945903956c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:31.634707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8901-0bcf-4de0-9455-b7945903956c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.636260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 117/221] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:31.641366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:31.780180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 105/222] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:31.788741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185c77f-fd66-47d6-9cce-ead8422e4cd2 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.788741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185c77f-fd66-47d6-9cce-ead8422e4cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.788741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185c77f-fd66-47d6-9cce-ead8422e4cd2 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.788741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 106/223] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:31.793441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185c77f-fd66-47d6-9cce-ead8422e4cd2 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:31.799017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:31.826091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f54c2-acf1-4e4d-b481-8b0bebe54a82 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.878794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd5dc3-1f46-43ce-81cf-340236e8f719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.880941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd5dc3-1f46-43ce-81cf-340236e8f719 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.881422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd5dc3-1f46-43ce-81cf-340236e8f719 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.881868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd5dc3-1f46-43ce-81cf-340236e8f719 None admin] PATH_INFO: `/v3/users/dccdda8cf5894e76a4bf0315694830eb` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.883782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd5dc3-1f46-43ce-81cf-340236e8f719 None admin] RBAC: Authorizing `identity:get_user(user_id=dccdda8cf5894e76a4bf0315694830eb)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:31.885456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd5dc3-1f46-43ce-81cf-340236e8f719 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.888180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 118/224] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/users/dccdda8cf5894e76a4bf0315694830eb => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:31.889970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.890259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.890536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.898993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:31.914818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:31.916548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:31.918194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185c77f-fd66-47d6-9cce-ead8422e4cd2 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:31.955473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:31.956353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:31.956647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:31.956949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:31.958231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:31.959607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:31.961805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 107/225] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:31.966655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.012279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 119/226] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:32.013584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.039029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:32.071805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f50742-b442-4048-b4b7-153e5b442577 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.110988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.111402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.111684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.111956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.113195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:32.114697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:32.116663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 108/227] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:31 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:32.117607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.147488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209fa8-563b-40b0-b7b6-5dc63d9bc898 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.183301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.184016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.184394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.184811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.185745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=25145350d12d4bfd8cdf5054e8832835) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:32.186209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:32.187835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:32.190295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:32.200665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 120/228] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:32 2022] GET /identity/v3/role_assignments?user.id=25145350d12d4bfd8cdf5054e8832835&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:32.200665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.280250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c181f1-8a31-4249-9306-b12d30c94388 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.350634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.351480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.351760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.352032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.370042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:32.374107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:32.387342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:32.388318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:32.389136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 109/229] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:32.401868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.402778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a518d-a973-48ad-8ed2-33383762e26a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.438575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.439490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.439772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.440047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/dccdda8cf5894e76a4bf0315694830eb/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.442230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] RBAC: Authorizing `identity:create_grant(project_id=42c6f87eff6c45a384b4edb237fbf44a, user_id=dccdda8cf5894e76a4bf0315694830eb, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:32.444202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:32.466076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 121/230] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:32 2022] PUT /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/dccdda8cf5894e76a4bf0315694830eb/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:32.544786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73410405-2263-4179-b59a-808ae72d2f99 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.587980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.588705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.588991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.589265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.590039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:32.590346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:32.591764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:32.606009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 110/231] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:32.613747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.712750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:32.759651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961643a-43df-4440-984d-52a2dab6e34b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.795304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.796541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.797065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.797557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] PATH_INFO: `/v3/users/dccdda8cf5894e76a4bf0315694830eb` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.799064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] RBAC: Authorizing `identity:get_user(user_id=dccdda8cf5894e76a4bf0315694830eb)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:32.800747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:32.802903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 122/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:32 2022] GET /identity/v3/users/dccdda8cf5894e76a4bf0315694830eb => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:32.808120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:32.884641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:32.931566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fe638-2ad5-4e90-be8e-0d5e39dd7f45 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:32.996145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49db2d2-228c-4bdc-843f-40e2a05e4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:32.996983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49db2d2-228c-4bdc-843f-40e2a05e4cba None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:32.997265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49db2d2-228c-4bdc-843f-40e2a05e4cba None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:32.997534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49db2d2-228c-4bdc-843f-40e2a05e4cba None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:32.998749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49db2d2-228c-4bdc-843f-40e2a05e4cba None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:33.000228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49db2d2-228c-4bdc-843f-40e2a05e4cba None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:33.002202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 111/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:32 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:33.008103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:33.161096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607471b5-352e-4f97-a610-a2fb6afdb8e6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:33.200194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:33.213540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e27d18-39d7-4540-b050-7549934eb6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:33.214782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e27d18-39d7-4540-b050-7549934eb6cc None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.215262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e27d18-39d7-4540-b050-7549934eb6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.215758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e27d18-39d7-4540-b050-7549934eb6cc None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/dccdda8cf5894e76a4bf0315694830eb/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.222622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e27d18-39d7-4540-b050-7549934eb6cc None admin] RBAC: Authorizing `identity:create_grant(project_id=42c6f87eff6c45a384b4edb237fbf44a, user_id=dccdda8cf5894e76a4bf0315694830eb, role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:33.224373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e27d18-39d7-4540-b050-7549934eb6cc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:33.258729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 123/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:33 2022] PUT /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/dccdda8cf5894e76a4bf0315694830eb/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:33.560123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f27c7-02e0-4d02-8268-2f0d7de080af None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.560837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f27c7-02e0-4d02-8268-2f0d7de080af None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.561325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f27c7-02e0-4d02-8268-2f0d7de080af None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.563158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 112/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:33.570813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.571370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.571865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.574963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe287f-0a4f-4c7e-8760-b7aee8be0607 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.575835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe287f-0a4f-4c7e-8760-b7aee8be0607 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.576858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe287f-0a4f-4c7e-8760-b7aee8be0607 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.579597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 113/236] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:33.584929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.585214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.585493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.595536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:33.597116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:33.605190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:33.606441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:33.680011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 124/237] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:33.689677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 114/238] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:33.704599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:33.714667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:33.773050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:33.812817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a02b3c-b90e-46ec-86e6-27e8fcf59112 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:33.823474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fa8d6-db9b-457e-b092-927b7d11f9ec None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:33.831072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:33.855425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:33.857498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.859851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.860522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.880753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:33.891542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:33.892617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:33.893215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:33.893679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:33.895662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:33.896552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:33.897407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:33.899135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:33.900845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:33.904377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 115/239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:33.904800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 125/240] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:33.909639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:33.915967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:34.031641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459cbc8-0d5f-49f9-a589-62fb3791ee61 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:34.069950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:34.077909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:34.078881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.080574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.082196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.083237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:34.086473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:34.136411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 116/241] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:33 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:34.143170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ac44-9262-4241-8a05-f6857765d4f7 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:34.183937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:34.184659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.184953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.185229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.186000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:34.186309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:34.187713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:34.197720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 126/242] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:34.203489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:34.312095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cefcc0-af90-42f1-8144-b072380b81fd None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:34.348591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:34.349465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.349750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.350025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] PATH_INFO: `/v3/users/8e76ebb1c9e44bc5aa8fe30642996309` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.365946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] RBAC: Authorizing `identity:get_user(user_id=8e76ebb1c9e44bc5aa8fe30642996309)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:34.367448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:34.369379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 117/243] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:34 2022] GET /identity/v3/users/8e76ebb1c9e44bc5aa8fe30642996309 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:34.375421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:34.419371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:34.485054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d9b8-0f8c-4b96-89ed-5e4388bd6a3a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:34.519445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9ebf2-0cc5-422d-821b-d24da663e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:34.520115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ebf2-0cc5-422d-821b-d24da663e1dd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.520399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ebf2-0cc5-422d-821b-d24da663e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.520706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ebf2-0cc5-422d-821b-d24da663e1dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.590438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9ebf2-0cc5-422d-821b-d24da663e1dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:34.591927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9ebf2-0cc5-422d-821b-d24da663e1dd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:34.632027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:34.695493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 127/244] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:34.701311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:34.703754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7973d66-e7aa-4a4e-9b5d-dbbd1ab58d40 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.704069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7973d66-e7aa-4a4e-9b5d-dbbd1ab58d40 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.704579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7973d66-e7aa-4a4e-9b5d-dbbd1ab58d40 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.707261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 128/245] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:34.716179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.716466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.717262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.740526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:34.741769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:34.753630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:34.815351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64df72-e86c-4ad3-8b1f-25775d3daa55 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:34.829695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 129/246] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:34.853498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:34.872070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:34.876066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:34.876687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:34.877500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:34.879024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:34.880744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:34.886021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 118/247] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:34 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:34.893824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:34.937711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:34.984307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05dfa1-b701-44fa-b899-57ba5a3ea1b1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.012678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f06ce8-c8bc-4470-b7f9-06913de89329 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.025693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:35.030790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.032906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.033429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.034140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.049361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:35.056874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.058141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.058568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:35.058720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:35.059743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:35.060523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:35.061155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 130/248] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:35.061767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.062858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] RBAC: Adding query filter params (role.id=060c3431b7d24ac98e46d6a8f9e4c556, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=8e76ebb1c9e44bc5aa8fe30642996309) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:35.063415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:35.065180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:35.068002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.073548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 119/249] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:34 2022] GET /identity/v3/role_assignments?user.id=8e76ebb1c9e44bc5aa8fe30642996309&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=060c3431b7d24ac98e46d6a8f9e4c556 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:35.184857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751e41f-8569-48af-a179-5285dca1bb16 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.214972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:35.224696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.225408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.225693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.225970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.226742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:35.227050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:35.228488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958a8c8-fdef-4493-b7e9-9dcbd176fffd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:35.239658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 131/250] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:35.245625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.254785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558f24c-19b1-45bc-93cd-226bac637340 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.255101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558f24c-19b1-45bc-93cd-226bac637340 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.255341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558f24c-19b1-45bc-93cd-226bac637340 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.261695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 132/251] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:35.267421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.267713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.267985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.289726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:35.291091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:35.373850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10050a00-fd77-49b3-b9a3-803818f9de81 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.380657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 133/252] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:35.400541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.421339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.422668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.423342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.423894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] PATH_INFO: `/v3/users/25145350d12d4bfd8cdf5054e8832835` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.425471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] RBAC: Authorizing `identity:get_user(user_id=25145350d12d4bfd8cdf5054e8832835)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:35.427144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:35.429310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 120/253] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:35 2022] GET /identity/v3/users/25145350d12d4bfd8cdf5054e8832835 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:35.450267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.516105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef7734-784d-4473-a1ac-6acd8f9fe6c1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.532516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:35.539595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:35.569857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.571287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.572437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.573085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.598238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:35.609908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:35.610962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:35.611939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:35.614730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 134/254] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:35.620921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.659087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496399a-429d-4439-a22b-0ab4ca452073 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.737488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.738776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.739272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.739744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.742203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:35.744024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:35.747283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 121/255] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:35 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:35.756651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.819169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660b48c-5b07-4f98-bafb-c23a7d08fb98 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.882094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:35.883163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:35.883663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:35.884127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:35.885243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:35.885742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:35.887630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:35.908358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 135/256] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:35.914625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:35.948707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d64a-cf04-4b20-89b2-288434d480a1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:35.969072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:36.013101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:36.037373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900820d7-cf33-4db0-a69f-a8e713531330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:36.038304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900820d7-cf33-4db0-a69f-a8e713531330 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.038586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900820d7-cf33-4db0-a69f-a8e713531330 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.038864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900820d7-cf33-4db0-a69f-a8e713531330 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/25145350d12d4bfd8cdf5054e8832835/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.045480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900820d7-cf33-4db0-a69f-a8e713531330 None admin] RBAC: Authorizing `identity:create_grant(project_id=42c6f87eff6c45a384b4edb237fbf44a, user_id=25145350d12d4bfd8cdf5054e8832835, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:36.046975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900820d7-cf33-4db0-a69f-a8e713531330 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:36.073350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 122/257] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:35 2022] PUT /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/25145350d12d4bfd8cdf5054e8832835/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:19:36.074793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea36e-63cf-4c22-9f12-7e5a3b0a63e4 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.075077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea36e-63cf-4c22-9f12-7e5a3b0a63e4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.075349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea36e-63cf-4c22-9f12-7e5a3b0a63e4 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.076768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 123/258] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:36.083303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.083616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.083853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.103520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:36.104833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:36.149980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd3e1e-e477-4013-a681-36ba6dd3f325 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:36.250025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:36.251238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.251624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.251896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] PATH_INFO: `/v3/users/dccdda8cf5894e76a4bf0315694830eb` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.253335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] RBAC: Authorizing `identity:get_user(user_id=dccdda8cf5894e76a4bf0315694830eb)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:36.256270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:36.262204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 136/259] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:35 2022] GET /identity/v3/users/dccdda8cf5894e76a4bf0315694830eb => generated 283 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:36.266577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 124/260] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:36.272089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:36.309285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:36.444438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737417f5-93b8-4023-b153-aa91a4b19780 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:36.464990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:36.517790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:36.518823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.519303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.519771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.565975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:36.569733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4e90-3c56-4c56-9694-56a2495ce0e0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:36.630950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:36.632452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:36.653904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:36.654749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.655026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.655293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.670802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:36.695870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:36.696943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:36.698395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 125/261] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:36.705576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:36.782907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 137/262] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:36.788852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:36.813007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:36.838918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04008825-31d9-4fa9-8458-83951b077a1e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:36.880098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:36.880804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:36.881086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:36.881358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:36.882124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:36.884650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:36.884650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:36.904862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 126/263] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:36.906105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:36.953055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:37.014824 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a825-15a8-4c86-a7d1-cd1197b087bd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:37.086542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015d349-80ff-4804-8ce5-196dc097aea2 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:37.098351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:37.099564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.100039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.100503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.101984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:37.103651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:37.105791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 138/264] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:36 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:37.113226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:37.186718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:37.187931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.188411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.188956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] PATH_INFO: `/v3/users/dccdda8cf5894e76a4bf0315694830eb` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.193350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] RBAC: Authorizing `identity:get_user(user_id=dccdda8cf5894e76a4bf0315694830eb)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:37.195447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:37.197394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 127/265] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:36 2022] GET /identity/v3/users/dccdda8cf5894e76a4bf0315694830eb => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:37.208980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:37.290094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:37.349371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9f966-05e8-4619-a4b4-0433ad8fd671 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:37.371757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:37.386932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503bfb3-a879-4bb6-ae07-6165fa4d2213 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:37.420371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:37.424983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.425280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.425560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.426421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=dccdda8cf5894e76a4bf0315694830eb) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:37.426736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:37.428198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:37.439405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 139/266] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:37 2022] GET /identity/v3/role_assignments?user.id=dccdda8cf5894e76a4bf0315694830eb&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:37.472142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2849337-ee64-419c-af4b-8a6cb107347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:37.473262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2849337-ee64-419c-af4b-8a6cb107347f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.473746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2849337-ee64-419c-af4b-8a6cb107347f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.474216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2849337-ee64-419c-af4b-8a6cb107347f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.582819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2849337-ee64-419c-af4b-8a6cb107347f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:37.584327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2849337-ee64-419c-af4b-8a6cb107347f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:37.703596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:37.719328 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 128/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:37.725893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:37.789521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81b642-76f1-44a3-bda4-4fa4fbb6aef2 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.790373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81b642-76f1-44a3-bda4-4fa4fbb6aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.791048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81b642-76f1-44a3-bda4-4fa4fbb6aef2 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.793676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 129/268] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:37.801718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13aefd-1022-4ac7-8a99-8486491c0090 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.802462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13aefd-1022-4ac7-8a99-8486491c0090 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.803126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13aefd-1022-4ac7-8a99-8486491c0090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.837668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7d13aefd-1022-4ac7-8a99-8486491c0090 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:37.839274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7d13aefd-1022-4ac7-8a99-8486491c0090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:37.919185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ffe3c-cb13-4bd3-b627-19df789efb35 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:37.952832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 130/269] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:37.954889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da12dc0-0b59-46be-8e0d-731df389eb11 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.955420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da12dc0-0b59-46be-8e0d-731df389eb11 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.955904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da12dc0-0b59-46be-8e0d-731df389eb11 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.957692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 131/270] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:37.963481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.964096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.964561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.977488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:37.978385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:37.978876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:37.979339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:37.981612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:37.983284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:37.988676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:37.988676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:37.989555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 140/271] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:37 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:37.995536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.010307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.084657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 132/272] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:38.084657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.142903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.144839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9559cd-50ed-44dd-b6ba-e5df90c53df2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.189736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.190578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.190720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.190933 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.191769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] RBAC: Adding query filter params (role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=dccdda8cf5894e76a4bf0315694830eb) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:38.192076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:38.193568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:38.200246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 141/273] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:37 2022] GET /identity/v3/role_assignments?user.id=dccdda8cf5894e76a4bf0315694830eb&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:38.208745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.218856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42015e-bec8-4d59-a8e5-c4fba9275566 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.289033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.289872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.290150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.290420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.291947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:38.293550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:38.295594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 133/274] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:38.301037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.351252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.378902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0283e-1a23-4a2a-a74b-bc4b5fb1d52d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.417515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7002c-542e-4488-9153-a39c924ac233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.419072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7002c-542e-4488-9153-a39c924ac233 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.419072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7002c-542e-4488-9153-a39c924ac233 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.419351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7002c-542e-4488-9153-a39c924ac233 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.434509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f179f33-cd87-47ea-8b1e-aefcc17621d4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.435523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7002c-542e-4488-9153-a39c924ac233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-a8d7002c-542e-4488-9153-a39c924ac233 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:38.445564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:38.446487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:38.447272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:38.447856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 142/275] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:38.449240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55bd54-e07b-4f46-a7d4-5088ea08544f None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.449531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55bd54-e07b-4f46-a7d4-5088ea08544f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.449807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55bd54-e07b-4f46-a7d4-5088ea08544f None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.452482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 143/276] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:38.453227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55bd54-e07b-4f46-a7d4-5088ea08544f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.460427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.486647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ff344-90b7-48fa-ade5-58a146a6814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.487861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ff344-90b7-48fa-ade5-58a146a6814d None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.487861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ff344-90b7-48fa-ade5-58a146a6814d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.488038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ff344-90b7-48fa-ade5-58a146a6814d None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.489921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ff344-90b7-48fa-ade5-58a146a6814d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:38.492778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ff344-90b7-48fa-ade5-58a146a6814d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:38.516086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 134/277] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:19:38 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:38.517834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.518543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.518974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.537416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:38.538935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:38.586987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55bd54-e07b-4f46-a7d4-5088ea08544f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.616238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 135/278] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:38.628985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.629925 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.630304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.630676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.631600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:38.632036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:38.633625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:38.635435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.641779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.648702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 144/279] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:38.653618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.759261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a83c-6103-4d1c-bb50-3faafe8ca275 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.762807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.783587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff77f76-57eb-45f7-bce9-b378eaa1d1f4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.804538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.805558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.805838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.806421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.824666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:38.830223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39655334-0550-4566-8571-c885700482ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:38.831467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39655334-0550-4566-8571-c885700482ca None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:38.831986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39655334-0550-4566-8571-c885700482ca None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:38.832470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39655334-0550-4566-8571-c885700482ca None admin] PATH_INFO: `/v3/users/8a2e8fa7d4bd481b9073c7cb39579d3d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:38.833213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:38.834079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:38.834851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:38.840967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 136/280] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:38.842263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.853880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39655334-0550-4566-8571-c885700482ca None admin] RBAC: Authorizing `identity:get_user(user_id=8a2e8fa7d4bd481b9073c7cb39579d3d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:38.855369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39655334-0550-4566-8571-c885700482ca None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:38.859491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 145/281] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/users/8a2e8fa7d4bd481b9073c7cb39579d3d => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:38.863712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39655334-0550-4566-8571-c885700482ca None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:38.933918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:38.969201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d86bd-45f2-4a37-b423-4f4c82c465cc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:38.987726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39655334-0550-4566-8571-c885700482ca None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.014137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.014896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.015183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.015789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.016901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:39.017215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.019097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740d2aa-4912-4777-b607-68cf0b14eea4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:39.027643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.028394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.028770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.029044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 137/282] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:39.029145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.030043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, user.id=8a2e8fa7d4bd481b9073c7cb39579d3d, scope.system=all) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:39.030358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fba5f8-c000-4d13-86fa-4a1f79066b90 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.030476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.030636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fba5f8-c000-4d13-86fa-4a1f79066b90 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.030916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fba5f8-c000-4d13-86fa-4a1f79066b90 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.032081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb1deb-be77-490d-80b5-6234cc0c003e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:39.032447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 138/283] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:39.034434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fba5f8-c000-4d13-86fa-4a1f79066b90 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:39.045559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 146/284] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jun 30 12:19:38 2022] GET /identity/v3/role_assignments?user.id=8a2e8fa7d4bd481b9073c7cb39579d3d&scope.system=all&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:39.046958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.047333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.047678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.087370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:39.088601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:39.098541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:39.248671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 147/285] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:39.254320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fba5f8-c000-4d13-86fa-4a1f79066b90 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.271073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:39.311348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.312242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.312522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.312824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] PATH_INFO: `/v3/users/8e76ebb1c9e44bc5aa8fe30642996309` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.314552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] RBAC: Authorizing `identity:get_user(user_id=8e76ebb1c9e44bc5aa8fe30642996309)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.316167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:39.318205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 139/286] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:39 2022] GET /identity/v3/users/8e76ebb1c9e44bc5aa8fe30642996309 => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:39.323719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:39.373355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:39.451265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3517-047d-4537-8ff1-ca31cad724f8 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.488233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.489131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.489415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.489742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.501923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:39.510796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:39.511680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:39.512516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 148/287] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:39.519942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:39.522303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:39.539527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f63eb3-5c5c-4252-a1b6-3bdc39e97f42 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.579388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.580341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.580646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.580922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.582144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.583621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:39.590268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 140/288] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:39 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:39.591625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:39.698737 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa29541-dc31-449a-ab21-73dfa07032e4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.700409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc32d0-f6b0-4e09-a829-d1645cf8c6f9 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.728961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:39.736894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.738002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.738292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.738561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/8e76ebb1c9e44bc5aa8fe30642996309/roles/060c3431b7d24ac98e46d6a8f9e4c556` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.740959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] RBAC: Authorizing `identity:create_grant(project_id=42c6f87eff6c45a384b4edb237fbf44a, user_id=8e76ebb1c9e44bc5aa8fe30642996309, role_id=060c3431b7d24ac98e46d6a8f9e4c556)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.742362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:39.758232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 141/289] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:19:39 2022] PUT /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/users/8e76ebb1c9e44bc5aa8fe30642996309/roles/060c3431b7d24ac98e46d6a8f9e4c556 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:39.800189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.800833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.801116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.801388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.802175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:39.802480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.803905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:39.823150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 149/290] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:39.829142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:39.865560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:39.957559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa66e4-f13b-431b-a901-729f8b6f3009 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:39.994596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:39.995520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:39.995794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:39.996070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] PATH_INFO: `/v3/users/25145350d12d4bfd8cdf5054e8832835` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:39.997491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] RBAC: Authorizing `identity:get_user(user_id=25145350d12d4bfd8cdf5054e8832835)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:39.998984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:40.001220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 142/291] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:39 2022] GET /identity/v3/users/25145350d12d4bfd8cdf5054e8832835 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:40.010022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:40.134516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c6854-6742-4634-a6ef-1865216fe46a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:40.170596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:40.170596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:40.172122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:40.172122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:40.215618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:40.245952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:40.247638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:40.348054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 150/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:40.353589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:40.463578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:40.473440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7d740-27b3-420a-b09f-baf6453edc58 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:40.508051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da308671-b048-4245-81e3-48e7c2b0aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:40.509203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da308671-b048-4245-81e3-48e7c2b0aa11 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:40.509626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da308671-b048-4245-81e3-48e7c2b0aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:40.510046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da308671-b048-4245-81e3-48e7c2b0aa11 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:40.511369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da308671-b048-4245-81e3-48e7c2b0aa11 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:40.513020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da308671-b048-4245-81e3-48e7c2b0aa11 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:40.515262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 143/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:40 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:40.521202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:40.592503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:40.624517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3daac4-1f76-456c-b55e-147e94c37a81 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:40.659411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:40.660429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:40.660950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:40.661413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:40.662440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=25145350d12d4bfd8cdf5054e8832835) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:40.662966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:40.664706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d5111-1fdc-489e-986f-2afbdd024e29 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:40.671389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 151/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:40 2022] GET /identity/v3/role_assignments?user.id=25145350d12d4bfd8cdf5054e8832835&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:41.068269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3404e-7a36-4f83-944a-5e462631c8ef None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.068671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3404e-7a36-4f83-944a-5e462631c8ef None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.069236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3404e-7a36-4f83-944a-5e462631c8ef None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.070628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 144/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:41.075861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9d04-916f-4aa2-a673-72637dc6421c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.076148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9d04-916f-4aa2-a673-72637dc6421c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.076419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9d04-916f-4aa2-a673-72637dc6421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.096854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7c5c9d04-916f-4aa2-a673-72637dc6421c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:41.098054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7c5c9d04-916f-4aa2-a673-72637dc6421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:41.181732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 152/296] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:41.208924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3404e-7a36-4f83-944a-5e462631c8ef None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.287643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9671f0-352a-4ba3-81d9-37ab33796d98 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.287963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9671f0-352a-4ba3-81d9-37ab33796d98 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.288198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9671f0-352a-4ba3-81d9-37ab33796d98 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.289645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 153/297] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:41.295219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.295508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.295790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.312948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:41.314628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:41.349461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3404e-7a36-4f83-944a-5e462631c8ef None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:41.384665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:41.395419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 154/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:41.395587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:41.396263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.396545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.396845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.407616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:41.415597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:41.416700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:41.417848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 145/299] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:41.421119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.422803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.457003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:41.532479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240710-c4f8-47f6-9a8c-f9da37430887 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:41.533603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c31b1-0021-41f1-96ce-b183f35dadce None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:41.568543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:41.569613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.569882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.570152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.570904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:41.571209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:41.571989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79406e39-644a-4f20-b8a6-cf84a5dc89d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:41.572649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:41.572892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79406e39-644a-4f20-b8a6-cf84a5dc89d7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.573177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79406e39-644a-4f20-b8a6-cf84a5dc89d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.573456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79406e39-644a-4f20-b8a6-cf84a5dc89d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.574825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79406e39-644a-4f20-b8a6-cf84a5dc89d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:41.576257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79406e39-644a-4f20-b8a6-cf84a5dc89d7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:41.578253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 155/300] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:41.585353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 146/301] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:41.585483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcd477-9d7b-443c-9f1c-6778e4c701e2 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.585483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcd477-9d7b-443c-9f1c-6778e4c701e2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.585483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcd477-9d7b-443c-9f1c-6778e4c701e2 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.585940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 156/302] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:41.586631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcd477-9d7b-443c-9f1c-6778e4c701e2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.589643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.649108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:41.683672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:41.689897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcd477-9d7b-443c-9f1c-6778e4c701e2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:41.706449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e7547-fcf9-46f4-b93d-4318df726ce6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:41.729305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19092b0c-7eec-47ec-807e-c430bb5672d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:41.729947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092b0c-7eec-47ec-807e-c430bb5672d5 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.730227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092b0c-7eec-47ec-807e-c430bb5672d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.730650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092b0c-7eec-47ec-807e-c430bb5672d5 None admin] PATH_INFO: `/v3/groups` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.731198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19092b0c-7eec-47ec-807e-c430bb5672d5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:41.732670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19092b0c-7eec-47ec-807e-c430bb5672d5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:41.741799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:41.743136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.743769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.744255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] PATH_INFO: `/v3/users/f10b7338000a47b599d2f341b62e4cf7` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.748043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 157/303] 10.222.0.57 () {60 vars in 1134 bytes} [Thu Jun 30 12:19:41 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:41.757401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.757758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.757954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.761344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] RBAC: Authorizing `identity:get_user(user_id=f10b7338000a47b599d2f341b62e4cf7)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:41.763169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:41.771111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 147/304] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:41 2022] GET /identity/v3/users/f10b7338000a47b599d2f341b62e4cf7 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:41.772584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.778023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:41.779307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:41.876828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 158/305] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:41.892480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458248b-aa8f-485f-91ac-7d200ba95b6f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:41.896857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:41.939510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:41.940191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:41.940474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:41.940809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:41.941775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] RBAC: Adding query filter params (role.id=060c3431b7d24ac98e46d6a8f9e4c556, user.id=f10b7338000a47b599d2f341b62e4cf7, scope.system=all) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:41.942084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:41.943594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:41.950855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 148/306] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jun 30 12:19:41 2022] GET /identity/v3/role_assignments?user.id=f10b7338000a47b599d2f341b62e4cf7&scope.system=all&role.id=060c3431b7d24ac98e46d6a8f9e4c556 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:41.963994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:42.027022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea9d54-2bc7-43b3-9f5a-6cacbb11533c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.066229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.067065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.067344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.067613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.080465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:42.088493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:42.089466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:42.090291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:42.095079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 159/307] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:42.096621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:42.099613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:42.210249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e33f7c-f997-4e44-bd4b-c7f340cf966b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.244972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.245660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.245940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.246211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.246965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] RBAC: Adding query filter params (name=member) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:42.247268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:42.248740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e5578-a986-4765-8d79-383902d29421 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:42.262127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 149/308] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:42.263634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:42.274282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c81f5-e510-47ce-8f8a-684d56b3f871 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.274565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c81f5-e510-47ce-8f8a-684d56b3f871 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.274848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c81f5-e510-47ce-8f8a-684d56b3f871 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.276494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 150/309] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:42.281496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.281784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.282062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.299985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:42.301246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:42.365434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:42.375148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 151/310] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:42.391706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec00e2-4078-44b4-ae6e-a8cb3bc22544 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.399769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:42.407632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:42.434914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e55703-8871-4add-b349-bb17912413fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.437054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e55703-8871-4add-b349-bb17912413fe None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.437339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e55703-8871-4add-b349-bb17912413fe None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.437617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e55703-8871-4add-b349-bb17912413fe None admin] PATH_INFO: `/v3/users/8a2e8fa7d4bd481b9073c7cb39579d3d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.441368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e55703-8871-4add-b349-bb17912413fe None admin] RBAC: Authorizing `identity:get_user(user_id=8a2e8fa7d4bd481b9073c7cb39579d3d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:42.443599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e55703-8871-4add-b349-bb17912413fe None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:42.451582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 160/311] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:42 2022] GET /identity/v3/users/8a2e8fa7d4bd481b9073c7cb39579d3d => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:42.453398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e55703-8871-4add-b349-bb17912413fe None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:42.511624 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccd32f-56d5-4e9a-83a7-9701446cdf4e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.548384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.549236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.549519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.549792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.561804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:42.566554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e55703-8871-4add-b349-bb17912413fe None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:42.570631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:42.571542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:42.572334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 152/312] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:42.582320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:42.608473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.609731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.610238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.610782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] PATH_INFO: `/v3/system/users/8a2e8fa7d4bd481b9073c7cb39579d3d/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.612514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a2e8fa7d4bd481b9073c7cb39579d3d, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:42.614177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:42.626961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 161/313] 10.222.0.57 () {58 vars in 1334 bytes} [Thu Jun 30 12:19:42 2022] PUT /identity/v3/system/users/8a2e8fa7d4bd481b9073c7cb39579d3d/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:19:42.657670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:42.707997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2e8ae-17dd-46d0-b492-1d2292f85b95 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.744745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.745449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.745730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.746008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.746795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:42.747104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:42.748504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:42.758484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 153/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:42.764158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:42.814514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:42.874673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e2ec9-8048-4854-84ca-699e171c90d3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:42.909257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:42.910147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:42.910431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:42.910705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] PATH_INFO: `/v3/users/8e76ebb1c9e44bc5aa8fe30642996309` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:42.911914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] RBAC: Authorizing `identity:get_user(user_id=8e76ebb1c9e44bc5aa8fe30642996309)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:42.913463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:42.915474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 162/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:42 2022] GET /identity/v3/users/8e76ebb1c9e44bc5aa8fe30642996309 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:42.921051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:43.027114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307f149-d796-461e-be0d-9b9ba532a20b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:43.044398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:43.064313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:43.065244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:43.065663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:43.066102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:43.139194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:43.140948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:43.245179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 154/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:43.250444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:43.362271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e298c7-5e66-4798-84a0-f383e1e640ed None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:43.375252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:43.397328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fe41d-4c37-4fc5-af34-1c37e46403ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:43.398206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe41d-4c37-4fc5-af34-1c37e46403ba None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:43.398485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe41d-4c37-4fc5-af34-1c37e46403ba None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:43.398756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe41d-4c37-4fc5-af34-1c37e46403ba None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:43.399942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fe41d-4c37-4fc5-af34-1c37e46403ba None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:43.401423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fe41d-4c37-4fc5-af34-1c37e46403ba None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:43.403140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 163/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:43 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:43.409493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:43.427645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:43.512300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824c56e-b1f0-4c1b-8baf-d3ef796ef9d2 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:43.547120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:43.547785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:43.548129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:43.548403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:43.549386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] RBAC: Adding query filter params (role.id=060c3431b7d24ac98e46d6a8f9e4c556, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a, user.id=8e76ebb1c9e44bc5aa8fe30642996309) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:43.549697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:43.551151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a6c74-8bd8-4f76-bca0-31cd659cd994 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:43.557442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 155/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:19:43 2022] GET /identity/v3/role_assignments?user.id=8e76ebb1c9e44bc5aa8fe30642996309&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=060c3431b7d24ac98e46d6a8f9e4c556 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:43.967467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f44a56-7800-4a78-a74b-ec14007e7285 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:43.968012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f44a56-7800-4a78-a74b-ec14007e7285 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:43.968012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f44a56-7800-4a78-a74b-ec14007e7285 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:43.969636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 164/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:43.975002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:43.975255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:43.975561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:43.992861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:43.994057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:44.015025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc43b32-d631-488d-9b40-e6e5c47dd8a5 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.015311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc43b32-d631-488d-9b40-e6e5c47dd8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.015589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc43b32-d631-488d-9b40-e6e5c47dd8a5 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.017303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 165/320] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:44.022122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.022417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.022644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.039233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:44.040403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:44.067804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 156/321] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:44.090579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.102660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:44.112816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 166/322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:44.131820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.202667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766d31c-192a-4616-b125-0d5be34bc64f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:44.238234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:44.238529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5c081-09df-4cdc-b4e5-036ac3c6ee71 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:44.238958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.239237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.239505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.240712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:44.242193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:44.244011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 157/323] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:44.249537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.251122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:44.277404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:44.278320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.278597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.278865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.289719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:44.298108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:44.299024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:44.299809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 167/324] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:44.305287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.335014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:44.357168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2257a9d-40e5-4b9c-8f47-2ee1a9e2a537 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:44.391319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:44.392055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.392375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.392753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] PATH_INFO: `/v3/groups` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.393449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:44.394918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:44.411282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 158/325] 10.222.0.57 () {60 vars in 1134 bytes} [Thu Jun 30 12:19:44 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:19:44.417777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685be96-a3a9-4930-aa33-59ad00ae68df None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:44.456172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:44.456894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.457174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.457470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.458233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:44.458541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:44.459983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:44.470146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 168/326] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:44.475952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.588317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9400b-f3dc-4dda-bd38-e4ce936e06ef None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:44.622659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09afeee-72dc-43fb-b8f4-6a2fcd7c72eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:44.623575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09afeee-72dc-43fb-b8f4-6a2fcd7c72eb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.623856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09afeee-72dc-43fb-b8f4-6a2fcd7c72eb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.624129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09afeee-72dc-43fb-b8f4-6a2fcd7c72eb None admin] PATH_INFO: `/v3/users/f10b7338000a47b599d2f341b62e4cf7` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.625377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09afeee-72dc-43fb-b8f4-6a2fcd7c72eb None admin] RBAC: Authorizing `identity:get_user(user_id=f10b7338000a47b599d2f341b62e4cf7)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:44.627103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09afeee-72dc-43fb-b8f4-6a2fcd7c72eb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:44.628805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 159/327] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:44 2022] GET /identity/v3/users/f10b7338000a47b599d2f341b62e4cf7 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:44.635027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.673163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:44.749792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dbe8f-24dd-4569-90b7-0661ef84dc77 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:44.785023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7278b9c-a678-497c-9b1a-662f39447f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:44.785908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7278b9c-a678-497c-9b1a-662f39447f3a None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.786256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7278b9c-a678-497c-9b1a-662f39447f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.786494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7278b9c-a678-497c-9b1a-662f39447f3a None admin] PATH_INFO: `/v3/system/users/f10b7338000a47b599d2f341b62e4cf7/roles/060c3431b7d24ac98e46d6a8f9e4c556` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.788083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7278b9c-a678-497c-9b1a-662f39447f3a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f10b7338000a47b599d2f341b62e4cf7, role_id=060c3431b7d24ac98e46d6a8f9e4c556)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:44.789495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7278b9c-a678-497c-9b1a-662f39447f3a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:44.800106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 169/328] 10.222.0.57 () {58 vars in 1334 bytes} [Thu Jun 30 12:19:44 2022] PUT /identity/v3/system/users/f10b7338000a47b599d2f341b62e4cf7/roles/060c3431b7d24ac98e46d6a8f9e4c556 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:44.848095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9de52-fd3f-4c99-a3c5-0db0ac05b45a None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.848281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9de52-fd3f-4c99-a3c5-0db0ac05b45a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.848563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9de52-fd3f-4c99-a3c5-0db0ac05b45a None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.850252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 160/329] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:44.856191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:44.856506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:44.856775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:44.875523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:44.877770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:44.961603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 170/330] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:44.986636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9de52-fd3f-4c99-a3c5-0db0ac05b45a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:44.996388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:45.100869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9de52-fd3f-4c99-a3c5-0db0ac05b45a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:45.135958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:45.136802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:45.137085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:45.137356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:45.148036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:45.156062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:45.157013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:45.157918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:45.159975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 161/331] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:45.166565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:45.205196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:45.286148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819a6da-1514-4d8d-8095-109414d83085 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:45.322257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:45.322944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:45.323225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:45.323501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:45.324259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:45.324656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:45.326266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:45.335656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 171/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:45.341240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:45.415581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:45.454058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b876ad-81bd-48a2-8768-139434482351 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:45.499631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3fc46-5e92-43b2-b380-6b2956392ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:45.500523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3fc46-5e92-43b2-b380-6b2956392ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:45.500827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3fc46-5e92-43b2-b380-6b2956392ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:45.501129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3fc46-5e92-43b2-b380-6b2956392ab2 None admin] PATH_INFO: `/v3/users/8a2e8fa7d4bd481b9073c7cb39579d3d` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:45.505388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3fc46-5e92-43b2-b380-6b2956392ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=8a2e8fa7d4bd481b9073c7cb39579d3d)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:45.506885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3fc46-5e92-43b2-b380-6b2956392ab2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:45.508943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 162/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:45 2022] GET /identity/v3/users/8a2e8fa7d4bd481b9073c7cb39579d3d => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:45.514735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:45.622674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:45.639109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d57e6-0ee0-42b2-bfb9-cdb5f322d7e7 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:45.675630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:45.676216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:45.676499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:45.676876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:45.677772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] RBAC: Adding query filter params (role.id=52504f5e779147cd8e9aa136bcd0ceae, user.id=8a2e8fa7d4bd481b9073c7cb39579d3d, scope.system=all) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:45.678092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:45.679563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811eedd-f069-40ce-908a-c3a83dccd57d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:45.693446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 172/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jun 30 12:19:45 2022] GET /identity/v3/role_assignments?user.id=8a2e8fa7d4bd481b9073c7cb39579d3d&scope.system=all&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:46.924895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a065b-0ca2-482a-bb90-95a4f6ddfd0a None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:46.925422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a065b-0ca2-482a-bb90-95a4f6ddfd0a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:46.925422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a065b-0ca2-482a-bb90-95a4f6ddfd0a None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:46.927173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 163/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:46.933118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401daf-518f-4393-b105-9b636b377975 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:46.933930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401daf-518f-4393-b105-9b636b377975 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:46.934348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401daf-518f-4393-b105-9b636b377975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:46.958228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-56401daf-518f-4393-b105-9b636b377975 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:46.959660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-56401daf-518f-4393-b105-9b636b377975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:47.019234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ab7c9-81c9-4512-bf75-d9e2ff0c2cbf None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.019748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ab7c9-81c9-4512-bf75-d9e2ff0c2cbf None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.020159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ab7c9-81c9-4512-bf75-d9e2ff0c2cbf None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.022007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 164/336] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:47.038564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.039208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.039678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.051561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 173/337] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:47.073603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:47.075112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:47.140668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56401daf-518f-4393-b105-9b636b377975 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:47.171159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 165/338] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:47.219929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:47.311461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:47.334101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:47.363163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161bd6f-961c-4b66-afdf-a09121b08223 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:47.411715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56401daf-518f-4393-b105-9b636b377975 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:47.423905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:47.425017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.425478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.425947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.442340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:47.451318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:47.452217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:47.457320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:47.459857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 166/339] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:47.465345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:47.500705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7de1b-9e41-48e5-90d0-c3313902a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:47.501687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7de1b-9e41-48e5-90d0-c3313902a561 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.501972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7de1b-9e41-48e5-90d0-c3313902a561 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.502246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7de1b-9e41-48e5-90d0-c3313902a561 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.520767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7de1b-9e41-48e5-90d0-c3313902a561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-02b7de1b-9e41-48e5-90d0-c3313902a561 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:47.531439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:47.532417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:47.533272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:47.535716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 174/340] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:47.537248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93508241-0ba5-482c-aff1-fd5cc127f7f9 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.537729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93508241-0ba5-482c-aff1-fd5cc127f7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.538175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93508241-0ba5-482c-aff1-fd5cc127f7f9 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.539631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 175/341] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:47.544528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93508241-0ba5-482c-aff1-fd5cc127f7f9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:47.676138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d2cfd-6889-4e07-bc12-6277b40c5f16 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:47.681080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93508241-0ba5-482c-aff1-fd5cc127f7f9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:47.727872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:47.750150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:47.751160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:47.752207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.753119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.753576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.754020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.754461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.755401 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:47.755866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.756976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:47.757319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:47.758916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a14d9c-cd84-4573-b2b3-05dedd48665d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:47.759425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:47.761030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:47.773020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:47.778078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 167/342] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:47.780225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 176/343] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:47.781857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:47.782370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:47.782847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:47.787914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:47.818634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:47.821568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:47.948956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 168/344] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:47.949822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:47.968685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a5873-3895-4f0d-b47d-8728abdbfc16 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.034481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.035059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.035337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.035609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.141819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea828-8770-4953-b16f-ffff16a1c477 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.176011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.182468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:48.183961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:48.198509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.224026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b421506-93f0-47d0-8839-c0e314bd47c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.224740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421506-93f0-47d0-8839-c0e314bd47c6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.225018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421506-93f0-47d0-8839-c0e314bd47c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.225285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421506-93f0-47d0-8839-c0e314bd47c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.317511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421506-93f0-47d0-8839-c0e314bd47c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:48.319449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421506-93f0-47d0-8839-c0e314bd47c6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:48.348746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 177/345] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:19:48.349971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.425386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 169/346] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:48.427074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be497245-dd11-48e4-b994-72095fbc41d1 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.427498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be497245-dd11-48e4-b994-72095fbc41d1 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.427909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be497245-dd11-48e4-b994-72095fbc41d1 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.429353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 170/347] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:48.430229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be497245-dd11-48e4-b994-72095fbc41d1 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.449441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.453493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.457724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6d425-3eb9-4186-bb98-d859e1ac9be9 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.494020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.494828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.495106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.495374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.505976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:48.513827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:48.514757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:19:48.515630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:48.519789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 178/348] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:48.520692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.544363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be497245-dd11-48e4-b994-72095fbc41d1 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.581236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f377-2138-4608-a544-ca152b064650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.583030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f377-2138-4608-a544-ca152b064650 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.583661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f377-2138-4608-a544-ca152b064650 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.584100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f377-2138-4608-a544-ca152b064650 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.586074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848f377-2138-4608-a544-ca152b064650 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:48.588588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848f377-2138-4608-a544-ca152b064650 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:48.591215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 171/349] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:48.593569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.594029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.594491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.616541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:48.618014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:48.627778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dcbce-dd58-42c4-9c0b-73dd23ba58dc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.662988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.664027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.664401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.664804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.666114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:48.667684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:48.669530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 179/350] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:48.670449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.696800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 172/351] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:48.698047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.700116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.716132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.779036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee575e-041c-4aa2-9fa1-0bfb39635375 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.811242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d5ab-f767-4649-ad9d-d4bf9a8b3bd6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.816641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.817353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.817631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.817904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.818658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:48.819340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:48.820748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:48.834693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 180/352] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:48.835545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.852090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:48.853013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:48.853295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:48.853569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:48.860820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:48.862398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:48.864149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 173/353] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:48.865463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:48.946436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:48.963520 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f3d7e-bbba-4709-a95c-f589e2b07e98 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:48.972247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:49.007686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.008578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.008988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.009266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.010525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:49.011342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f59bd-e709-4bc8-a13a-9d2473717efa None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.011990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:49.014753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 181/354] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:49.015735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.059943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd938f1-1a70-4672-afe9-74a3fa42bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.060799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd938f1-1a70-4672-afe9-74a3fa42bbbd None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.061285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd938f1-1a70-4672-afe9-74a3fa42bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.061738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd938f1-1a70-4672-afe9-74a3fa42bbbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.082420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd938f1-1a70-4672-afe9-74a3fa42bbbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-afd938f1-1a70-4672-afe9-74a3fa42bbbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:49.087316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:49.088208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:49.089072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 174/355] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:49.090248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65e91f-b4c5-4eb4-a60f-0465bfb564ee None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.090939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65e91f-b4c5-4eb4-a60f-0465bfb564ee None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.090939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65e91f-b4c5-4eb4-a60f-0465bfb564ee None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.092218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 175/356] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:49.101431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65e91f-b4c5-4eb4-a60f-0465bfb564ee None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.152453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748309-cce3-43b1-9cb4-c1d7eb144d6f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.198943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.200273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.200553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.200866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] PATH_INFO: `/v3/users/f10b7338000a47b599d2f341b62e4cf7` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.203333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] RBAC: Authorizing `identity:get_user(user_id=f10b7338000a47b599d2f341b62e4cf7)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:49.204974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:49.206690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 182/357] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/users/f10b7338000a47b599d2f341b62e4cf7 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:49.207424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.242203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65e91f-b4c5-4eb4-a60f-0465bfb564ee None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.245027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:49.287380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.288495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.288873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.289151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.290108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:49.290418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:49.291896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c84354-c208-4dbf-a13d-c9298717fd69 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:49.298338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 176/358] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:49.300707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.300997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.301271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.306686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:49.321294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:49.324240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:49.347850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b2bc4-dda8-4cf7-bf39-08dc16efee00 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.404984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.405672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.405947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.406216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.407066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:49.407734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:49.410437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:49.419913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 183/359] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:49.420896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.432312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 177/360] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:49.433356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.570318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cb531-a3f1-4903-9eda-50f943c08867 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.571162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:49.585757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afd262-2f92-49b0-8af1-cd870dad0275 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.609872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:49.617070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.617886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.618169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.618445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.619828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] RBAC: Adding query filter params (role.id=060c3431b7d24ac98e46d6a8f9e4c556, user.id=f10b7338000a47b599d2f341b62e4cf7, scope.system=all) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:49.620152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:49.622087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae0702-3828-4269-a88b-b150b3fa6ec6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:49.627729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.629010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.629399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.629895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.630758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:49.631170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:49.633748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c90a7c-812d-4046-939b-6e57ef252e95 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:49.637268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 178/361] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/role_assignments?user.id=f10b7338000a47b599d2f341b62e4cf7&scope.system=all&role.id=060c3431b7d24ac98e46d6a8f9e4c556 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:49.638454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ba61-90a1-4a5c-b94c-145e032da902 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.638683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ba61-90a1-4a5c-b94c-145e032da902 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.638958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ba61-90a1-4a5c-b94c-145e032da902 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.641126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 179/362] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:49.642317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ba61-90a1-4a5c-b94c-145e032da902 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.644231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 184/363] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:49.650348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.650684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.651041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.671902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:49.673476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:49.790344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 185/364] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:49.791422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.869654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ba61-90a1-4a5c-b94c-145e032da902 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:49.920496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:49.921375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:49.921660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:49.921936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:49.934313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:49.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:49.945058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:49.945886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:49.948046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 180/365] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:49.949090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:49.952412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:49.976776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1abe9f-2565-4784-9b0f-282bb3e7cd87 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.023852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.026079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.026480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.026578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.027357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.098940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59e92-7c56-4ec4-8ec8-18940bc139d8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.119251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:50.121083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:50.144272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.145363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.145775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.146703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.161230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:50.172528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:50.174375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:50.175358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:50.178083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 181/366] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:50.179639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.224343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.260746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 186/367] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:50.265811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.321459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28357797-ea42-4e3b-baf0-682c6267cfd0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.335611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.366928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.368152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.368579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.369021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.370599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:50.371056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:50.373176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:50.385091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 182/368] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:50.386682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.394470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446498-265a-49b6-b5d3-f821124f43d3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.444281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.445664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.445947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.446293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.446966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:50.447271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:50.448883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:50.460324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 187/369] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:50.460967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.513074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.519958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71da89-8882-41d1-905c-7a12a1883bfb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.562555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.563495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.563882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.564468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.565492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:50.566878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:50.569429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 183/370] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:50.575858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.584316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad024e2-e7e6-43dd-bdc2-b13f31c3d47c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.627088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.628923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.630404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.630694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.630967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.701084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4600e58-1a96-48af-811a-e3c0eb07a825 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:50.726951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:50.728578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:50.743692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:50.744369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:50.744800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:50.745073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:50.781705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.838447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:50.839963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:50.857438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 188/371] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:50.858715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.889069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:50.958742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 184/372] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:50.959939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:50.974252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423540a-ac4b-48ad-a632-fb8e123675d7 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.014185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.015008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.015282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.015550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.019011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.020673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.027364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 189/373] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.028816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.043705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:51.087174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e59dc0-fc81-49ec-803f-801742521ee5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.132847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.134106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.134588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.135071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.136545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.138245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.140562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 185/374] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:50 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.141665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.154337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:51.163091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df25c2-577b-4198-8b95-85b491126ba3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.199997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.201009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.201289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.201566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] PATH_INFO: `/v3/projects/e7a9e0309e534d4980ce851f75758fe8` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.204693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] RBAC: Authorizing `identity:get_project(project_id=e7a9e0309e534d4980ce851f75758fe8)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.206430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.208315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 190/375] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/projects/e7a9e0309e534d4980ce851f75758fe8 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.209139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.261437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a94af-39c7-44ed-8c53-a39167582997 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.297746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:51.301848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.302797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.303285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.303751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.304838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:51.305357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.307051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.315082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 186/376] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.316404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.321560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a38ff8-127a-4c2b-b7dd-3f9a9b930411 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.363532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.364433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.364749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.365026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.366846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.368033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.369970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 191/377] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.376360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.415508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:51.458095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23b9a9-d33a-4111-a6cb-7907095ef2d6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.496290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d4fd0-f96b-4aec-8640-c46e22a78ec4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.535899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.536571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.536903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.537181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.537338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d45f-63b7-4746-9357-fd9761a2f2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.538016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:51.538225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d45f-63b7-4746-9357-fd9761a2f2ee None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.538385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.538545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d45f-63b7-4746-9357-fd9761a2f2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.538778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d45f-63b7-4746-9357-fd9761a2f2ee None admin] PATH_INFO: `/v3/groups/587ced8e7196484a93928f7c0128f54b` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.539890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.546592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 192/378] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:51.549417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d45f-63b7-4746-9357-fd9761a2f2ee None admin] RBAC: Authorizing `identity:get_group(group_id=587ced8e7196484a93928f7c0128f54b)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.550939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d45f-63b7-4746-9357-fd9761a2f2ee None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.553059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 187/379] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/groups/587ced8e7196484a93928f7c0128f54b => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.559167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.581114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:51.717365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d39da-51b5-4d31-8af0-2ca73a85d652 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:51.764659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:51.765356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.765639 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.765913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.766839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] RBAC: Adding query filter params (group.id=587ced8e7196484a93928f7c0128f54b, role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=e7a9e0309e534d4980ce851f75758fe8) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:51.767183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:51.768706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75f54d-d8a2-4de5-87d0-1368c06e0244 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:51.775415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 193/380] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/role_assignments?group.id=587ced8e7196484a93928f7c0128f54b&scope.project.id=e7a9e0309e534d4980ce851f75758fe8&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:51.850339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8ef12-3768-42be-adbd-eb14bbe4da9c None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.850664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8ef12-3768-42be-adbd-eb14bbe4da9c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.850892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8ef12-3768-42be-adbd-eb14bbe4da9c None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.851872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11149b7-2fb9-4446-81ed-7e3243746c23 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.852160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11149b7-2fb9-4446-81ed-7e3243746c23 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.852437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11149b7-2fb9-4446-81ed-7e3243746c23 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.852686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 188/381] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:51.854025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 194/382] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:51.858219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.858511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.858783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.859397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:51.859683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:51.859967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:51.877994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:51.879222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:51.883178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:51.884409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:51.963719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 195/383] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:51.989796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:51.994309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:52.035592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 189/384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:52.072953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.104220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc032c-cb60-4ee8-a470-7e7ebd6a9cb8 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.147126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.148303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.148760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.149510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.164776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:52.171067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:52.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:52.178718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:52.179699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:52.182071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 196/385] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:52.188790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.220932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acff31-444c-442b-ac00-1b3416e86557 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.263386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:52.264306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.265166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.265598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.266017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.279955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:52.290739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:52.291687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:52.292500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 190/386] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:52.300650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.316757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfac59-b1c0-4b5c-baed-83667484446f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.364377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.365101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.365384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.365661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.366428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:52.366737 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:52.368139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:52.377668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 197/387] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:52.389992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.431165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452bc06-46ac-4564-b20e-345cbdcd6b5a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.470011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.471044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.471830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.472110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.473125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:52.473441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:52.475580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:52.485520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 191/388] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:52.485814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:52.491340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.610545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b037f2-76d7-460f-a533-b73e9d548fd6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.650166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.651730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.652287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.652849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.654425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:52.656569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:52.659029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fc175-6e79-4cfa-a7fe-5452a6a7846d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.660008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 192/389] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:52.667954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.714267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:52.737992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:52.745342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.746255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.746528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.746808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.748033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:52.750353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:52.751537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 198/390] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:52.758631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.794079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182b31c-3f7c-4d85-ba0a-232eac0e267e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.839454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.842928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.843414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.847062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.848567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:52.850250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:52.854583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 193/391] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:52.860371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:52.945871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353981e0-e066-4812-af13-e05b24e92fbb None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:52.986575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:52.987822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:52.988260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:52.988786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:52.990241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:52.991867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:52.994189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 199/392] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:52 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:52.999581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:53.020223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44e4dc-6f18-4322-b9f9-41e1c72653c0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:53.063666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:53.086828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564949-6cb6-4fce-aa6c-57afeb3a83e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:53.087680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564949-6cb6-4fce-aa6c-57afeb3a83e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:53.087957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564949-6cb6-4fce-aa6c-57afeb3a83e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:53.090460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564949-6cb6-4fce-aa6c-57afeb3a83e8 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/groups/68ce86e0081b4d468c5408104dc6cb77/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:53.091295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:53.094187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03564949-6cb6-4fce-aa6c-57afeb3a83e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbe7793149f98e33576b094aa8a2, group_id=68ce86e0081b4d468c5408104dc6cb77, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:53.096394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03564949-6cb6-4fce-aa6c-57afeb3a83e8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:53.121275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 194/393] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jun 30 12:19:52 2022] PUT /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/groups/68ce86e0081b4d468c5408104dc6cb77/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:19:53.143649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab606-3187-4d02-9821-0e5f38c59db2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:53.188350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8620c-8a5f-453e-874c-7b5ce81d734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:53.189832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8620c-8a5f-453e-874c-7b5ce81d734f None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:53.190116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8620c-8a5f-453e-874c-7b5ce81d734f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:53.190512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8620c-8a5f-453e-874c-7b5ce81d734f None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/groups/68ce86e0081b4d468c5408104dc6cb77/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:53.192928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8620c-8a5f-453e-874c-7b5ce81d734f None admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbe7793149f98e33576b094aa8a2, group_id=68ce86e0081b4d468c5408104dc6cb77, role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:53.194921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8620c-8a5f-453e-874c-7b5ce81d734f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:53.214617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 200/394] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jun 30 12:19:52 2022] PUT /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2/groups/68ce86e0081b4d468c5408104dc6cb77/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:54.300956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b59d35-2fac-4390-84e4-4e287afc332c None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.301490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b59d35-2fac-4390-84e4-4e287afc332c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.301490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b59d35-2fac-4390-84e4-4e287afc332c None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.303102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 195/395] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:54.308506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.309373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.309857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.331183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:54.332771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:54.337409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f234a-df67-412c-979b-99733424a98c None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.337957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f234a-df67-412c-979b-99733424a98c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.338427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f234a-df67-412c-979b-99733424a98c None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.340244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 196/396] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:54.348713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.349269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.349737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.381626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:54.383201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:54.414718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 201/397] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:54.461633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:54.501215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 197/398] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:54.519318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:54.541299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:54.575452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75cd8f-9a32-4b9f-8273-a596dd706fb8 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:54.630551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:54.631758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.632254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.632742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.654857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:54.666470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:54.667422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:54.668313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:54.676048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 202/399] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:54.687129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:54.748530 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112f4f3-4773-4d1f-aa2b-b534bf3ac326 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:54.765868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:54.792996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f14db-777d-4c4e-948e-4a4a1c9ae4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:54.794768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f14db-777d-4c4e-948e-4a4a1c9ae4ed None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.795524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f14db-777d-4c4e-948e-4a4a1c9ae4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.795993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f14db-777d-4c4e-948e-4a4a1c9ae4ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.810157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f14db-777d-4c4e-948e-4a4a1c9ae4ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9b5f14db-777d-4c4e-948e-4a4a1c9ae4ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:54.822377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:54.823295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:54.824087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:54.826163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 198/400] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:54.827285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fd07e-c825-4a56-a349-e75eaedc7682 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.827569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fd07e-c825-4a56-a349-e75eaedc7682 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.827850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fd07e-c825-4a56-a349-e75eaedc7682 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.832719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 199/401] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:54.833488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fd07e-c825-4a56-a349-e75eaedc7682 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:54.836199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa333cdc-9858-4ffa-b1cb-3002dfd73c51 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:54.850782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:54.876331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:54.877567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.877950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.878326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.879171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:54.879575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:54.881082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7acd4a-fe0a-45d1-9972-80ec9c82e8ff None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:54.890192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 203/402] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:54.891499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:54.896311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:54.896688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:54.913173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:54.914497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:54.961892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fd07e-c825-4a56-a349-e75eaedc7682 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:54.985246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 204/403] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:54.986383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.006347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.007312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.008292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.008773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.009980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:55.010728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.013106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.023677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 200/404] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:55.029279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.078526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.109047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f9097-3a63-4fe9-a893-93428065ef8e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.112486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.155157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88f4b8-1ff2-4c26-bd39-02df8bd4d6de None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.165089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.166756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.167649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.168409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.170141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.172617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.175493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 205/405] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:54 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:55.176930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.198835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.199706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.199981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.200252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.201505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.202989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.205243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 201/406] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:55 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:55.206104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.305248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730f39e-ee0e-45be-9f6c-70f60b48ac34 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.324897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b1ff8-6a68-4cb6-b40d-d2842e7dd593 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.328020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.352220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.353745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.354224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.354741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.361595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.362469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.362748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.363020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.364323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.365827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.368415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:55.372447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 202/407] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:55 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:55.373327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:55.377021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:55.377936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:55.378736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:55.382047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 206/408] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:55.387449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.422245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.485668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dfded-ffc1-4dc8-acac-cad295cc323b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.513692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0900877-a890-468e-98b4-a8328ad9ced4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.539368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.542008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.543465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.543966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.546338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.549240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.553005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 203/409] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:55 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:55.553976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.571997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.574642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.574642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.574642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/groups/68ce86e0081b4d468c5408104dc6cb77/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.577302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] RBAC: Authorizing `identity:create_grant(project_id=42c6f87eff6c45a384b4edb237fbf44a, group_id=68ce86e0081b4d468c5408104dc6cb77, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.580125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.611395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 207/410] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jun 30 12:19:55 2022] PUT /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/groups/68ce86e0081b4d468c5408104dc6cb77/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:55.612537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.627772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.731152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cc53b-6c35-45f4-9de3-bdf0ae940342 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.770505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.772124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.773222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.773497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.774271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:55.774585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.775977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3827af-2815-4cb7-b803-113d69e91f8f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.790731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 204/411] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:55.792956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21873b4-84da-45b6-812d-f45ae275d62d None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.793244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21873b4-84da-45b6-812d-f45ae275d62d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.793527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21873b4-84da-45b6-812d-f45ae275d62d None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.794811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 205/412] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:55.796943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b400ab-5e00-407b-929d-c897dd0ddd3e None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.797232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b400ab-5e00-407b-929d-c897dd0ddd3e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.797511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b400ab-5e00-407b-929d-c897dd0ddd3e None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.799246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 206/413] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:55.800036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b400ab-5e00-407b-929d-c897dd0ddd3e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:55.807191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.823681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309d848-f049-436c-9587-fd19fb27a32b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.866511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ca058-2d8e-4aa3-a2b9-e913838f2b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:55.867680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca058-2d8e-4aa3-a2b9-e913838f2b57 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.867962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca058-2d8e-4aa3-a2b9-e913838f2b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.868386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca058-2d8e-4aa3-a2b9-e913838f2b57 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/groups/68ce86e0081b4d468c5408104dc6cb77/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.871375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ca058-2d8e-4aa3-a2b9-e913838f2b57 None admin] RBAC: Authorizing `identity:create_grant(project_id=42c6f87eff6c45a384b4edb237fbf44a, group_id=68ce86e0081b4d468c5408104dc6cb77, role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:55.873454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ca058-2d8e-4aa3-a2b9-e913838f2b57 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:55.891951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 208/414] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jun 30 12:19:55 2022] PUT /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a/groups/68ce86e0081b4d468c5408104dc6cb77/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:55.893295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2d0a9-3c00-4f2c-ae0b-e24142f80885 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:55.893584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2d0a9-3c00-4f2c-ae0b-e24142f80885 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:55.893863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2d0a9-3c00-4f2c-ae0b-e24142f80885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:55.912325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4ec2d0a9-3c00-4f2c-ae0b-e24142f80885 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:55.913660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4ec2d0a9-3c00-4f2c-ae0b-e24142f80885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:55.969485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b400ab-5e00-407b-929d-c897dd0ddd3e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:55.971973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:55.999519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 209/415] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:56.001308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.001589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.001865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.020527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:56.022161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:56.051052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.052588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.052905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.053184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] PATH_INFO: `/v3/groups/587ced8e7196484a93928f7c0128f54b` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.054609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] RBAC: Authorizing `identity:get_group(group_id=587ced8e7196484a93928f7c0128f54b)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:56.056713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:56.058828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 207/416] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:55 2022] GET /identity/v3/groups/587ced8e7196484a93928f7c0128f54b => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:56.059646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.100383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 210/417] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:56.101239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.117018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:56.203488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7321e1-fcc0-46c6-9dd4-79dda68f9cf0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.219316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec723d09-e7ae-465a-8eb3-a67b61872f0c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.250552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.251398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.251682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.251958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.258591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.259907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.260365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.260863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] PATH_INFO: `/v3/projects/e7a9e0309e534d4980ce851f75758fe8` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.262359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] RBAC: Authorizing `identity:get_project(project_id=e7a9e0309e534d4980ce851f75758fe8)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:56.264052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:56.264369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:56.266624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 211/418] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/projects/e7a9e0309e534d4980ce851f75758fe8 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:56.267823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:56.282123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:56.283134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:56.283911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 208/419] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:56.284687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.325683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:56.372238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:56.398292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6162f8-1df3-43a9-aa1f-c5417c09d7ac None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.420796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa46b19-7f7e-4cbc-a1f1-756e52b971bc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.441508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.442335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.442979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.442979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.454494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:56.459206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.460270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.460726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.461139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] PATH_INFO: `/v3/projects/e7a9e0309e534d4980ce851f75758fe8/groups/587ced8e7196484a93928f7c0128f54b/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:56.462976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:56.464081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:56.465105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:56.466086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7a9e0309e534d4980ce851f75758fe8, group_id=587ced8e7196484a93928f7c0128f54b, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:56.467673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:56.468968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 212/420] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:56.469809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.493238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 209/421] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jun 30 12:19:56 2022] PUT /identity/v3/projects/e7a9e0309e534d4980ce851f75758fe8/groups/587ced8e7196484a93928f7c0128f54b/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:19:56.494387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.632218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:56.654296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73682333-6c44-44e9-8ec6-35ac4839e0f0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.666115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c632a62-c037-4f1b-90dd-706a0c4e8465 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.722325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.723114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.723410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.723787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.724553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] RBAC: Adding query filter params (name=member) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:56.724924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:56.725509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.726196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.726354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:56.726497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.726739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.727504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:56.728029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:56.729806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:56.739874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 213/422] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:56.745449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.746638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 210/423] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:56.750873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:56.752651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:56.859913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bab86-15f1-482d-bff5-3c51f68b685b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.911116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:56.911692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:56.911981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:56.912254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:56.941034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:56.962612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0225f10-ea16-480e-976a-9000abe7345d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:56.981811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:56.983319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.044427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:57.045117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.045397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.045668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.122054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 211/424] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:19:57.125365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:57.198946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:57.205073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:57.206662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.209433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:57.269379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb84c2e-1256-4d3b-a46c-f32bf349a1a5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:57.312114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:57.313018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.313298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.317673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.319313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:57.320802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.322533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 212/425] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:57 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:57.328290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:57.345309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 214/426] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:19:57.357637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:57.485830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5785df-313b-47c2-8b15-35d7839a5867 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:57.487567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:57.497003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50032716-d74b-481f-a6b0-75ee1f9ac6c5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:57.530165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:57.547788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:57.548878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.549157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.549426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] PATH_INFO: `/v3/projects/1fa9bbe7793149f98e33576b094aa8a2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.550924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] RBAC: Authorizing `identity:get_project(project_id=1fa9bbe7793149f98e33576b094aa8a2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:57.552381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.554405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 215/427] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:57 2022] GET /identity/v3/projects/1fa9bbe7793149f98e33576b094aa8a2 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:57.556608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:57.557511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.557795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.558066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.568649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:57.568649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.568649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 213/428] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:57 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:57.570247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:57.572481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:57.755651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba57412-e0da-417e-905d-8d641ae88b55 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:57.762645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d8314-1a74-4384-943b-409b4c7ee523 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:57.801732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:57.802409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.802684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.802966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.803801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:57.804111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:57.805588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.816309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 214/429] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:57 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:57.839546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e609a34-bc5e-42b0-a09f-7efaa81d3fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:57.840458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e609a34-bc5e-42b0-a09f-7efaa81d3fda None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.840779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e609a34-bc5e-42b0-a09f-7efaa81d3fda None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.841056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e609a34-bc5e-42b0-a09f-7efaa81d3fda None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.842383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e609a34-bc5e-42b0-a09f-7efaa81d3fda None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:57.843901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e609a34-bc5e-42b0-a09f-7efaa81d3fda None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:57.845868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 216/430] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:57 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:57.851961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:57.924713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:57.958546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608425c-1a41-4087-bad1-dcb24e103cb6 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.959177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608425c-1a41-4087-bad1-dcb24e103cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.959649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608425c-1a41-4087-bad1-dcb24e103cb6 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.961646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 217/431] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:57.967106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f38c6-8d82-44ae-bcf1-6d62125ef877 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:57.967635 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f38c6-8d82-44ae-bcf1-6d62125ef877 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:57.968106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f38c6-8d82-44ae-bcf1-6d62125ef877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:57.987759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-359f38c6-8d82-44ae-bcf1-6d62125ef877 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:57.989358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-359f38c6-8d82-44ae-bcf1-6d62125ef877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:58.010254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256d5c3-b7c5-4f44-925d-3c14bbdfa272 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.048598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:58.073891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.074809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.075254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.075677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.076680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=1fa9bbe7793149f98e33576b094aa8a2) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:58.077147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:58.078720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:58.087756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 215/432] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:57 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=1fa9bbe7793149f98e33576b094aa8a2&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:58.098602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 218/433] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:19:58.122018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.244150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47ab3f-11ff-4c9f-85c9-bc2f9a040d46 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.268323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3401f56-3dfd-4504-9350-2e0afba31a00 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.268496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3401f56-3dfd-4504-9350-2e0afba31a00 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.268802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3401f56-3dfd-4504-9350-2e0afba31a00 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.269722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:58.270554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 219/434] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:19:58.276776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.277399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.279085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.294343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.295221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.295504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.295774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.300359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:58.301818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:58.309356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:58.317332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:58.318264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:58.319404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 216/435] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:58.325063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.383187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 220/436] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:58.407047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.455208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813a33-77c4-47a5-adad-41130b4fbc90 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.491837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.492519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.492854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.493126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.493881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:58.494186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:58.495563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:58.505655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 217/437] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:58.510978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.532047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:58.540570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33403eca-7c96-4b92-a8ac-517e71457208 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.557644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:58.588039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.588907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.589356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.589461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.606842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:58.615039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:58.615927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:19:58.616944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:19:58.621953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 221/438] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:19:58.622841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.640699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f20ef-f8a2-458a-adad-64d71692db0a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.683478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe47923-e586-42a5-b61b-96f7cff33dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.684131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe47923-e586-42a5-b61b-96f7cff33dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.684414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe47923-e586-42a5-b61b-96f7cff33dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.684712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe47923-e586-42a5-b61b-96f7cff33dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.751413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07e187-6622-4ac9-9a5b-29cdc7dccb11 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.763985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe47923-e586-42a5-b61b-96f7cff33dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:58.765496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe47923-e586-42a5-b61b-96f7cff33dc5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:58.788751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.789349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.789631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.789904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.790651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:58.791215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:58.792346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:58.801833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 222/439] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:58.809545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.820426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:58.842013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:58.898966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 218/440] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:19:58.900300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c511b3-a919-4212-a82e-d6c31a04fc73 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.900587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c511b3-a919-4212-a82e-d6c31a04fc73 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.900883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c511b3-a919-4212-a82e-d6c31a04fc73 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:58.904690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 219/441] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:19:58.904690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c511b3-a919-4212-a82e-d6c31a04fc73 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:58.928267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac69df-869f-4237-a7f7-1d66d32d94b2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:58.962037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:58.962697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:58.962978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:58.963288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.003609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c511b3-a919-4212-a82e-d6c31a04fc73 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.032310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.033840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.038063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106cf81-bb74-4194-bac4-278f03c6f222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.038971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106cf81-bb74-4194-bac4-278f03c6f222 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.039476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106cf81-bb74-4194-bac4-278f03c6f222 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.039750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106cf81-bb74-4194-bac4-278f03c6f222 None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.040950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106cf81-bb74-4194-bac4-278f03c6f222 None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.042390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106cf81-bb74-4194-bac4-278f03c6f222 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.044168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 220/442] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:19:59.048808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.048808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.048808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.056098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.058881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.062321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:19:59.063534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:19:59.127710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 221/443] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:19:59.128541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.137687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 223/444] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:19:59.143903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.229672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c865-fb8f-43c1-96b7-c2bf00c7d298 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.257173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c38281-2891-474d-8eb9-ac3e3fd8403d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.263843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.264720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.265008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.265272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.266417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.268241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.269927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 222/445] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:59.270646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.274094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.283489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.298043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.298952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.299232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.299496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] PATH_INFO: `/v3/projects/42c6f87eff6c45a384b4edb237fbf44a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.300723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] RBAC: Authorizing `identity:get_project(project_id=42c6f87eff6c45a384b4edb237fbf44a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.302180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.303854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 224/446] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/projects/42c6f87eff6c45a384b4edb237fbf44a => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:59.304601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.367888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37008651-fc50-46d7-9abf-c166e4b0ff29 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.401394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.402185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.402467 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.402734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.407862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9970b-1275-4e37-a03f-24dc2e87195e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.413044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:19:59.420649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:19:59.421518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:19:59.422329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 223/447] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:19:59.422979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.442990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.443653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.444271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.444552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.445423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=52504f5e779147cd8e9aa136bcd0ceae, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:59.445735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.447180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.453079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 225/448] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=52504f5e779147cd8e9aa136bcd0ceae => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:59.453848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.490226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.526743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.536008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016d437-93a7-4e23-91ac-fd091f08b72c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.574516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.575667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.575952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.576227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] PATH_INFO: `/v3/groups/68ce86e0081b4d468c5408104dc6cb77` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.577607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] RBAC: Authorizing `identity:get_group(group_id=68ce86e0081b4d468c5408104dc6cb77)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.579306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.586086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 224/449] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/groups/68ce86e0081b4d468c5408104dc6cb77 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:59.586281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372699dc-bc4c-4600-933b-c07ad9f7025f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.587306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.627324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.628010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.628294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.628565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.629347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:59.629653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.631059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.640332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 226/450] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:59.645957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:19:59.702386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbe2c-d94a-4630-bba9-2197bec60b6d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.734039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.742144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.742834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.743113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.744058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.744730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] RBAC: Adding query filter params (group.id=68ce86e0081b4d468c5408104dc6cb77, role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=42c6f87eff6c45a384b4edb237fbf44a) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:19:59.745694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.748140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:19:59.753321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 225/451] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/role_assignments?group.id=68ce86e0081b4d468c5408104dc6cb77&scope.project.id=42c6f87eff6c45a384b4edb237fbf44a&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:19:59.769926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3078e-ce01-4842-8e8a-9f57f55da2fb None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:19:59.787813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:19:59.815588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:19:59.816305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:19:59.816589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:19:59.816898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:19:59.893767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:19:59.895164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:00.000471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 227/452] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:20:00.006611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:00.116012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f43b51-e468-4641-a1c3-9a28c363e858 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:00.149177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:00.149983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:00.150263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:00.150540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] PATH_INFO: `/v3/projects/e7a9e0309e534d4980ce851f75758fe8` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:00.151719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e7a9e0309e534d4980ce851f75758fe8)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:00.153184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:00.154933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 226/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:20:00 2022] GET /identity/v3/projects/e7a9e0309e534d4980ce851f75758fe8 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:00.161522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:00.183405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:00.273614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e378-b41b-4bb3-bceb-8d8bd315171d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:00.309750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7449d-9b1c-4324-8f4f-ed2cc04aefdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:00.310606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7449d-9b1c-4324-8f4f-ed2cc04aefdd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:00.310879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7449d-9b1c-4324-8f4f-ed2cc04aefdd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:00.311141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7449d-9b1c-4324-8f4f-ed2cc04aefdd None admin] PATH_INFO: `/v3/groups/587ced8e7196484a93928f7c0128f54b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:00.312406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7449d-9b1c-4324-8f4f-ed2cc04aefdd None admin] RBAC: Authorizing `identity:get_group(group_id=587ced8e7196484a93928f7c0128f54b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:00.314043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7449d-9b1c-4324-8f4f-ed2cc04aefdd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:00.315669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 228/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jun 30 12:20:00 2022] GET /identity/v3/groups/587ced8e7196484a93928f7c0128f54b => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:00.324774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:00.388556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:00.429736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153f5c3-db47-4021-9178-225d093297a9 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:00.465957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:00.466606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:00.466888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:00.467159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:00.467989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] RBAC: Adding query filter params (group.id=587ced8e7196484a93928f7c0128f54b, role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=e7a9e0309e534d4980ce851f75758fe8) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:00.468303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:00.469771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb052d-a176-4936-82b6-0d72125d399c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:00.475696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 227/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jun 30 12:20:00 2022] GET /identity/v3/role_assignments?group.id=587ced8e7196484a93928f7c0128f54b&scope.project.id=e7a9e0309e534d4980ce851f75758fe8&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:03.058080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1871bff-5181-464e-a387-3f301453fdcd None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.058943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1871bff-5181-464e-a387-3f301453fdcd None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.059368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1871bff-5181-464e-a387-3f301453fdcd None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.061183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 229/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:03.074420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab44ed2-8eca-4337-a10a-cc124d25f74b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.075238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab44ed2-8eca-4337-a10a-cc124d25f74b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.075651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab44ed2-8eca-4337-a10a-cc124d25f74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.094767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2ab44ed2-8eca-4337-a10a-cc124d25f74b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:03.096430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2ab44ed2-8eca-4337-a10a-cc124d25f74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:03.199283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 228/457] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:03.224349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1871bff-5181-464e-a387-3f301453fdcd None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.287628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1c505-fc1d-47b4-b251-b09efb880e62 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.288146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1c505-fc1d-47b4-b251-b09efb880e62 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.288561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1c505-fc1d-47b4-b251-b09efb880e62 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.290489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 229/458] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:20:03.301523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62512d-2a79-4996-ab51-9d282c5f3308 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.302104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62512d-2a79-4996-ab51-9d282c5f3308 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.302677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62512d-2a79-4996-ab51-9d282c5f3308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.326232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0b62512d-2a79-4996-ab51-9d282c5f3308 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:03.327627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0b62512d-2a79-4996-ab51-9d282c5f3308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:03.343317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:03.368545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1871bff-5181-464e-a387-3f301453fdcd None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:03.400119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:03.406088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:03.407163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.408344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.408858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.420236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 230/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:03.423389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b283-2020-4b11-8c3c-e5615231318d None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.424396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b283-2020-4b11-8c3c-e5615231318d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.424883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b283-2020-4b11-8c3c-e5615231318d None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.426692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 231/460] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:03.428661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:03.430356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:03.433791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.435010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.435494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.462520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:03.463519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:03.464345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:03.465203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:03.465203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:03.465203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:03.465203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:03.465432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:03.467455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 230/461] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:03.473696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.552732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 232/462] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:03.553947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.594973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50318c-fa72-49f1-97c2-1c227e976193 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:03.639608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:03.640850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.641378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.641650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.646172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:03.659136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:03.661426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:03.675826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76ac77-bc8e-462c-88fe-5106e65b376e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:03.685117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:03.685983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:03.686806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:03.686806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:03.686806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:03.686806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:03.688851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 231/463] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:03.690217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.708137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:03.728787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:03.729933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.730485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.730942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.731877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:03.732463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:03.734258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:03.752740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 233/464] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:03.754059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.826465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6cc65-1aad-417a-afca-57680b452569 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:03.867323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a6a5-1aa1-4098-ac59-0afb463da085 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:03.874334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:03.875153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.875838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.876112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.887483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:03.889048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:03.902512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:03.903202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.903482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.903750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.904464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:03.904803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:03.906154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eafab2-7672-4796-a606-ff7c58f097f9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:03.906667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:03.907617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:03.908494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:03.908494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:03.908494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:03.908494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:03.912867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 232/465] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:03.913637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.920617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 234/466] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:03.922370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdfbda-b0db-4ec3-b7b9-eea0956c4a46 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:03.922722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdfbda-b0db-4ec3-b7b9-eea0956c4a46 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:03.923128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdfbda-b0db-4ec3-b7b9-eea0956c4a46 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:03.924490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 235/467] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:03.925335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdfbda-b0db-4ec3-b7b9-eea0956c4a46 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:03.960333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:03.972591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.023128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212983f4-001d-4e61-bb61-810b79ca311a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.040310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdfbda-b0db-4ec3-b7b9-eea0956c4a46 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.059937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.060593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.060909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.061172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.061861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.063283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.078245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.078916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.079195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.079515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.080351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:04.080690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.082073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1ae95-32ce-4f43-b8fa-72ff388ced69 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.086124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 233/468] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:20:03 2022] POST /identity/v3/users => generated 248 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:04.087135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.109147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 236/469] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:04.111129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.111556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.111950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.133235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:04.135377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:04.209631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.217795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 237/470] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:04.218455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.218574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f77c21-0aad-4f97-850a-ed84757022a0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.253364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.263306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9ffc3-0840-43ad-aa04-86190db226dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.263995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9ffc3-0840-43ad-aa04-86190db226dc None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.264300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9ffc3-0840-43ad-aa04-86190db226dc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.264567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9ffc3-0840-43ad-aa04-86190db226dc None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.265309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9ffc3-0840-43ad-aa04-86190db226dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.268390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9ffc3-0840-43ad-aa04-86190db226dc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.290316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 234/471] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:20:04 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:04.291710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a4a44-95ae-430f-b654-1405064a5a56 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.291999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a4a44-95ae-430f-b654-1405064a5a56 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.292344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a4a44-95ae-430f-b654-1405064a5a56 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.293612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 235/472] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:04.295117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a4a44-95ae-430f-b654-1405064a5a56 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.350379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e39821-82ba-4b90-b028-c1c02f7d69d0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.401214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7ca9b-4c01-4dc7-bb3a-c2259501b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.401822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7ca9b-4c01-4dc7-bb3a-c2259501b27c None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.402104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7ca9b-4c01-4dc7-bb3a-c2259501b27c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.402380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7ca9b-4c01-4dc7-bb3a-c2259501b27c None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.403105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7ca9b-4c01-4dc7-bb3a-c2259501b27c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.405118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7ca9b-4c01-4dc7-bb3a-c2259501b27c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.433280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 238/473] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:20:04 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:04.434641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.434930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.435389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.437147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a4a44-95ae-430f-b654-1405064a5a56 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.458539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:04.460292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:04.479687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.481112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.481605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.482075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.495939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.498102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:04.519346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:04.520312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:04.521607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:04.521607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:04.521607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:04.521607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:04.524125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 236/474] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:04.531687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.536923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.550750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 239/475] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:04.577913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.596599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.689000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eceb184-9807-496d-aa95-ee20b4b29506 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.708853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86cf2-fcc8-4e46-8296-cb3cb295c1b4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.735956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.737460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.737753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.738149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.738902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:04.739214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.740725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.756276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 237/476] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:04.762013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.768592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.769522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.769954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.770327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.771243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.772870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.779460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.sql.core [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 12:20:04.779460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 12:20:04.779460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [parameters: {'id': 'f80eb0e2fb00417381efd2ec9496e0af', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 12:20:04.779460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80097) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application result.read() Jun 30 12:20:04.797363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:20:04.798404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application yield resource Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise self.value Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application session.commit() Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 12:20:04.799302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self.session.flush() Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application compat.raise_( Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 12:20:04.800153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:20:04.801029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application result.read() Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:20:04.801831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [parameters: {'id': 'f80eb0e2fb00417381efd2ec9496e0af', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:04.802596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 12:20:04.803362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:04.804294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 240/477] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jun 30 12:20:04 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:04.807263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:04.838014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.880086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448d5f6-368d-4622-84d9-e4a65e7ffc54 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.914450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bb9b9-17f8-438d-91ce-0aed4fb99c9d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:04.916583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.917438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.917870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.918259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.919096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:04.920704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:04.931195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:04.953251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 238/478] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:20:04 2022] POST /identity/v3/users => generated 249 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:20:04.972823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477acec-2d7e-4bd7-8283-3edf7a3095e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:04.973852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477acec-2d7e-4bd7-8283-3edf7a3095e5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:04.974231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477acec-2d7e-4bd7-8283-3edf7a3095e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:04.974604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477acec-2d7e-4bd7-8283-3edf7a3095e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:04.997294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477acec-2d7e-4bd7-8283-3edf7a3095e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a477acec-2d7e-4bd7-8283-3edf7a3095e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:05.011317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:05.012525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:20:05.013395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:05.014327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 241/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:05.020445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:05.164372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9cea5-3ac5-4024-83fa-71a37b80590d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:05.193754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:05.214431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:05.215546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:05.216242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:05.216573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:05.217413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:05.217777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:05.219226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03799bec-d4e2-4485-afcf-7008ab00843b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:05.229294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 239/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:06.800420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e149d80-a372-4b3d-9de7-c8bc01ae5b6b None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:06.800972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e149d80-a372-4b3d-9de7-c8bc01ae5b6b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:06.800972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e149d80-a372-4b3d-9de7-c8bc01ae5b6b None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:06.802784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 242/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:06.808494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:06.808947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:06.809123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:06.833821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:06.835035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:06.840508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdb924-50fa-4228-b241-9777cb25ea4b None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:06.841080 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdb924-50fa-4228-b241-9777cb25ea4b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:06.841557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdb924-50fa-4228-b241-9777cb25ea4b None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:06.843342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 243/482] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:06.854389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:06.854679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:06.855134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:06.884281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:06.885522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:06.948705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 240/483] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:06.974693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:06.998470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 244/484] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:07.035632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.075574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e755238-a3a1-4d54-a1ce-e61856bf0e41 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.086241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.112182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a273b-e360-4656-a23b-c6248d0cfaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.113038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a273b-e360-4656-a23b-c6248d0cfaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.113317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a273b-e360-4656-a23b-c6248d0cfaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.113588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a273b-e360-4656-a23b-c6248d0cfaa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.123046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.134670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a273b-e360-4656-a23b-c6248d0cfaa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-1f0a273b-e360-4656-a23b-c6248d0cfaa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:07.145621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:07.146486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:07.147266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:07.150356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 241/485] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:07.151821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44756687-e3f6-4379-88d4-71c772b03e2c None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.152245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44756687-e3f6-4379-88d4-71c772b03e2c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.152652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44756687-e3f6-4379-88d4-71c772b03e2c None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.158254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 242/486] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:07.158254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44756687-e3f6-4379-88d4-71c772b03e2c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.169556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002f31b-9676-4d39-8106-a9b470a968a4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.206015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b948-1c39-4a35-b6bb-cbdbd15fcd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.206986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b948-1c39-4a35-b6bb-cbdbd15fcd0b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.207352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b948-1c39-4a35-b6bb-cbdbd15fcd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.207714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b948-1c39-4a35-b6bb-cbdbd15fcd0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.218934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461b948-1c39-4a35-b6bb-cbdbd15fcd0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-4461b948-1c39-4a35-b6bb-cbdbd15fcd0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:07.227291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:07.228247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:07.229095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:07.232876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 245/487] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:07.234207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.234555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.235140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.253056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:07.254336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:07.282966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44756687-e3f6-4379-88d4-71c772b03e2c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.319467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.320451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.321061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.321567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.322571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:07.323348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.324954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.329032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 246/488] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:07.330002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.339017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 243/489] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:07.346000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.385096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.406507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.450705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60981e-68d4-443b-b74c-d59160ed7b18 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.461481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97be31b-7a1a-4c31-bda0-39e55892828d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.485851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.486536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.486855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.487111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.487919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:07.488221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.489676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.499103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.500228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.500733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.501236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.502823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 247/490] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:07.503797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.513947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.515803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:07.534622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:07.535545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:07.536382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:07.536382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:07.536382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:07.536382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:07.542946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 244/491] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:07.544716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.616982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacbd66-35b2-42e1-a8e8-2f6e3ff7a49f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.634580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.637200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.652827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.653627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.653903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.654173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.656311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7c5f-3611-4df7-8b04-c97afa8381ad None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.665395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:07.673226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:07.674329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:07.675114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:07.675709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 248/492] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:07.676393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.697618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.698426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.698726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.698971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.710443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.711920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f30bd0c7-41fe-4c5e-8df9-3d7931f818c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:07.729611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:07.730558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:07.731406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:07.731406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:07.731406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:07.731406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:07.731406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 245/493] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:07.732320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12a5c-40da-4129-9626-46c35dc4872c None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.732320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12a5c-40da-4129-9626-46c35dc4872c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.732559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12a5c-40da-4129-9626-46c35dc4872c None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.733842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 246/494] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:07.737197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c12a5c-40da-4129-9626-46c35dc4872c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.787207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab32a-1e56-466b-876f-cbd0e77d2b33 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.821431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.822098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.822381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.822657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.823386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:07.823698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.825126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.837295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 249/495] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:07.838092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.840053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c12a5c-40da-4129-9626-46c35dc4872c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.878804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.878804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.879103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.879103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.879843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:07.880156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.880464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.881664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6421b-5b24-4886-9b8d-efd913aeeb7a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.885808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:07.893100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 247/496] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:07.894352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.894633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.894903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.918978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:07.920219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:07.941471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9961cc-b243-46e2-9981-f0ac1580c989 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:07.976668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:07.977341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.977617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.977889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.978615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:07.978919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:07.980297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2f60-28e4-4832-a17d-31d758d907b7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:07.991508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 250/497] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:07.991930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 248/498] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:07.992829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:07.994084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab232a-e109-4f8b-b466-187818cf1937 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:07.994583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab232a-e109-4f8b-b466-187818cf1937 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:07.995126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab232a-e109-4f8b-b466-187818cf1937 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:07.996499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 251/499] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:07.997412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab232a-e109-4f8b-b466-187818cf1937 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.093337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.094534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd963e6-4e57-4353-ace4-fa908a24b2ce None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.097868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab232a-e109-4f8b-b466-187818cf1937 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.121858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.130548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.130709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.131282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.131564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.131662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.131855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.131950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.132111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.139581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.141446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.142430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.143931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:08.156926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:08.158244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 249/500] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:08.159142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-800f19c5-ad42-44a6-8f65-8e82d6354978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:08.160600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:08.161405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:08.162231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:08.162231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:08.162231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:08.162231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:08.162231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 252/501] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:08.162985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.163260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.163533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.182424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:08.183635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:08.250966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 253/502] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:08.251972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.281404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104ea06-a27c-4e97-845e-9c99fd9ef9d6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.316404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.317238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.317522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.317794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.325184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.326663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.338155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:08.340888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:08.341767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:08.342674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 250/503] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:08.343234 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.362063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090b3d4-023e-45ae-ad27-509674b6f893 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.393575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.395334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.396168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.396445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.396749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.407893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:08.415695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:08.416650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:08.417465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 254/504] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:08.418636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.449016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0eea3-a3e7-4f89-b001-22dc5470fdf3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.482481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.483153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.483434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.483707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.484536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:08.484874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.486481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.517185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148fbc-354a-43b8-a614-ba7e4405dbb5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.522327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 251/505] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:08.522974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.553083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.553754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.554036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.554307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.555041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:08.555348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.556782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.568599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 255/506] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:08.569766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.572528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.623523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fd58c-92cb-44fb-85b3-320d7fdc4bc1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.630727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.658488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.659249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.659534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.659803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.669842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.671363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.677504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fee3-b335-40de-9ed7-e9ab8fd06f0d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:08.689419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:08.690452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:08.691596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:08.691596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:08.691596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:08.691596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:08.691596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 252/507] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:08.692103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.719178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.719856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.720292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.720755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.721611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:08.721915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.723343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.759800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 256/508] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:08.760427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.792307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c544e5-9554-4a07-a41a-d0e5eafabdf4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.818871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.825348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.826018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.826301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.826576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.827309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:08.827617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.829055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.837081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 253/509] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:08.837896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:08.860096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:08.866770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8c93-a677-4f93-918d-e1984f8adb63 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.900721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.901396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.901679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.901952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.940944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b5111-cee2-4343-935a-0beb96406b5e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:08.969615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.971297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.975122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:08.975922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:08.976207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:08.976482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:08.984006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:08.985498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:08.999679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:09.000616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:09.001549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 254/510] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:09.002247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.033682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.074896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 257/511] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:09.075718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.084862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.104400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6c7d8-16f6-4ac1-9888-34a88c594c65 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.136806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.137475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.137758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.138033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.138761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:09.139071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.140476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.152590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 255/512] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:09.153569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.176939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf69898-fe36-4dc8-9990-64d38cf85929 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.211316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.211893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.212172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.212443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.239582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.254783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98be2-2d28-432a-888f-bed47b6ada1b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.279030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.280523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.290048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.290847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.291396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.291396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.299524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.302230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.303710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:09.322494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:09.323512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:09.324375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:09.324375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:09.324375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:09.324375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:09.324375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 256/513] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:09.324670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.385167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 258/514] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:09.386150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.431043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04a79-eb59-4e4f-af33-36506728654c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.446974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.469353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.469995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.470566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.470566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.471462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:09.471767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.473266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.488426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa37bb7-e44d-4f05-a4ef-0293aa96b5df None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.490426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 257/515] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:09.491252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.522756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.523650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.523996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.524274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.534195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.535652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.543342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:09.553647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:09.554633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:09.555661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:09.555661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:09.555661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:09.555661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:09.555661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 259/516] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:09.555661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.597722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871229ff-a5f4-4572-86e3-e76c81818bbf None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.630868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.631544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.631829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.632104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.632837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.634304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.654368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 258/517] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:20:09 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:09.654617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295b45-85d2-4830-8fc7-861e98f3cbe7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.655166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.669153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.695051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.695824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.696174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.696527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.697349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:09.697720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.699214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.712716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 260/518] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:09.713639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.784051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afb545-8a91-479f-8177-1bca9c3513f6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.791540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.827827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274a31-0ea4-4cda-a6f5-420c0045c69a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:09.829025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.829922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.830203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.830474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.842998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.844549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:09.863370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:09.864866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:09.865707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:09.865707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:09.865707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:09.865707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:09.865707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 259/519] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:09.865707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:09.874985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:09.875646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:09.875926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:09.876266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:09.920229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:09.949156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:09.950661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:09.976742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5843351-9b14-40c0-91ea-04988e7d42c0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.010449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.011126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.011410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.011685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.012413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:10.012759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.014145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.030572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 260/520] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:10.030572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.049272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.059897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 261/521] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:10.064526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.136209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.142624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854c72-b7ee-4713-a449-c071c27fcaa0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.179948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1bdc-df26-4613-876c-0e246d1346ef None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.183265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.184078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.184352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.184618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.194321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.195778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:10.210695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:10.211611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:10.212497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 261/522] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:10.213844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.220400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.221103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.221384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.221661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.222388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:10.222694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.224110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.239957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 262/523] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:10.239957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.308830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.333706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2170e6-d6b5-4577-bad1-71fcb2a1741b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.340865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a8d97-4e9a-471f-af3f-dd8c3ede11cb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.370716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.371522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.371654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.371794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.372066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.378635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.379531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.379814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.380096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.381078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.383169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.391460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.393584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:10.397467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:10.398413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 262/524] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:10.399811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:10.416407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:10.417340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:10.418172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:10.418172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:10.418172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:10.418172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:10.418172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 263/525] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:10.418541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.509110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550bc0-eba0-40f5-a138-4661d3843179 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.524463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15ca5c-5dee-44dd-bd2c-06711181f609 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.543673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.544377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.544658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.544929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.545765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:10.546350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.547803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.558559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.559116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.559347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.559623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.559923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.567469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.568945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.573478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 263/526] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:10.574159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:10.585583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:10.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:10.587298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 264/527] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:10.588421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.637774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.688252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbff64d-1b69-45a0-ab7d-b5d4d6d7d7a1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.698192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572aafc-106d-49ca-84ea-3f33f0ac5ff5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.724317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.725019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.725309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.725584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.726367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:10.726678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.728136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.735137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.735847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.736125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.736398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.737155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:10.737462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.739329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.750751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 264/528] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:10.750751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.761538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 265/529] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:10.761538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.814086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.859620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce600b9-41c4-4a42-8e51-e87cf2f96e3a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.879081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be601d94-aa69-43d5-9e86-4375f3ca6157 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:10.882123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:10.894613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.895272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.895551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.895926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.921450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:10.922124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:10.922466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:10.922954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:10.923521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:10.923828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.925528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:10.949344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 266/530] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:10.949344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:10.973047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:10.974509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.029607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:11.062715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff1421-d6ca-4c02-ab1c-8a7cad44468e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.075378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 265/531] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:11.076167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.103967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.104961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.105442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.105907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.106950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0ca71f32f61e41288cac452e58b4a1dc) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:11.107475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.109222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.115373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 267/532] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:10 2022] GET /identity/v3/role_assignments?user.id=0ca71f32f61e41288cac452e58b4a1dc&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:11.116408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.159669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:11.202583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab31d7-5489-47ce-a4f8-e8c31fe46be1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.243382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.244585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.245130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.245606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.253665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.255558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.267541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a94df-7a5d-49af-a7ff-dcd9cfaae4c2 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:11.271303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:11.272182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:11.273045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:11.275290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 266/533] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:11.281243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.304037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:11.309156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.309828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.310662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.310953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.311807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=a8e81a1c18d34adf8d8a45fa09f26099) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:11.312121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.314009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.324713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 268/534] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/role_assignments?user.id=a8e81a1c18d34adf8d8a45fa09f26099&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:11.325958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.429264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdb99f-0c2a-4030-a19d-f5fde3e21b1f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.502388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380781-3305-42f9-9979-f69738a45963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.503218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380781-3305-42f9-9979-f69738a45963 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.503502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380781-3305-42f9-9979-f69738a45963 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.503786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380781-3305-42f9-9979-f69738a45963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.520692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:11.527104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98380781-3305-42f9-9979-f69738a45963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.528610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98380781-3305-42f9-9979-f69738a45963 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.536053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e240-8a0c-4b5a-9934-f700513b9fb4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-98380781-3305-42f9-9979-f69738a45963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:11.560901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:11.561818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:11.563090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:11.563090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:11.563090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:11.563090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:11.563090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 267/535] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:11.570589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380781-3305-42f9-9979-f69738a45963 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.611661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.612220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.612504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.612796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.613633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:11.613902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.615350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.633840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 269/536] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:11.639942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.684815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:11.752190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380781-3305-42f9-9979-f69738a45963 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.793952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.795173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.795462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.795741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.796470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:11.796874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.797336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0f835-1a24-44a9-b6e5-57cf046b1b24 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:11.798377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649270f-7e90-42eb-80f8-9d2da14a969d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.813670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 268/537] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:11.815486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ee294-a88b-497f-8ac6-3de4675360d1 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.815910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ee294-a88b-497f-8ac6-3de4675360d1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.816310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ee294-a88b-497f-8ac6-3de4675360d1 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.818030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 269/538] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:11.819576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ee294-a88b-497f-8ac6-3de4675360d1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:11.844737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:11.845559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.845898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.846772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.847836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=d61b8631879a461db96399a9b91682c0) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:11.848315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:11.850637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a213b5d-4f99-4d7d-9a9f-f8ed63b69b58 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:11.861248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 270/539] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/role_assignments?user.id=d61b8631879a461db96399a9b91682c0&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:11.862790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:11.863081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:11.863444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:11.882803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:11.884701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:11.901746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:11.973700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 271/540] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:12.014185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:12.034583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ee294-a88b-497f-8ac6-3de4675360d1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.093782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:12.103371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.104231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.104513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.104815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.118609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:12.120209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:12.137680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:12.139733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:12.141844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 270/541] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:12.150882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:12.169541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30771130-842f-4c07-9a11-d2bcb2965465 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.228674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.229491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.229769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.230038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.251807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:12.259645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:12.261702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:12.263637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 272/542] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:12.273334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:12.297881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:12.308951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab7008-fc19-4b7a-bff7-05b214d5813b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.376982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.377557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.377839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.378111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.378918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:12.379221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:12.380706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:12.409523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 271/543] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:12.415542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:12.489377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4aeae-d5de-471f-bc25-56c9dd31ec45 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.517441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:12.540471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.541177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.541457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.541729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.542486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:12.542800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:12.544160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:12.552989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 273/544] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:12.566015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:12.578803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb5173-d3d2-4a14-8b2c-de6b008c8079 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.654775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.655804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.656291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.656795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.657979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=94b91e199622457787fa2a7e78d743c6) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:12.658669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:12.660338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:12.661828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:12.667962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 272/545] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:12 2022] GET /identity/v3/role_assignments?user.id=94b91e199622457787fa2a7e78d743c6&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:12.706325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efbeb-02fc-4ec4-83dd-cea7b3c32b0b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.751762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.752612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.752942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.753210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.765799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:12.768104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:12.803479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:12.804406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:12.805288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:12.805288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:12.805288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:12.805288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:12.805288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 274/546] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:12.810666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:12.953863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc010486-1539-47a2-9125-775b9a9dad8b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:12.975872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:12.997554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:12.998231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:12.998513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:12.998788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:12.999511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:12.999816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:13.001278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:13.013476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 273/547] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:13.019066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:13.136706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd705984-200a-4bff-a3b5-5085740036a9 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:13.165411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:13.175847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:13.176680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.176968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.177242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.185420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:13.186894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:13.203575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:13.205043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:13.206443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 275/548] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:13.211382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:13.379435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a0690-2a29-429a-93dd-7d5e0eb9f9e5 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:13.424417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:13.425521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.426014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.426511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.427604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:13.428119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:13.429783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60d40e-67c4-45fe-892a-5eaaa601f0f5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:13.457915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 274/549] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jun 30 12:20:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:13.463079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:13.505728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb728a-6fc1-404a-bd3f-0583682044ea None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.506487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb728a-6fc1-404a-bd3f-0583682044ea None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.506868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb728a-6fc1-404a-bd3f-0583682044ea None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.508649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 275/550] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:13.513870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.514280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.514658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.540680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:13.541899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:13.592995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:13.649738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86faa42-4786-42cb-8ce5-b3acd19708f8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:13.672460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 276/551] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:13.695437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:13.696099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.696380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.696667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.703994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:13.766536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:13.793143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:13.794636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:13.826161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d074c-6e9b-450e-9193-550db3d49ba4 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:13.861984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46f9d-fc72-48a3-8c18-4c554a465e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:13.862791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46f9d-fc72-48a3-8c18-4c554a465e17 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.863080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46f9d-fc72-48a3-8c18-4c554a465e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.863354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46f9d-fc72-48a3-8c18-4c554a465e17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.873826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46f9d-fc72-48a3-8c18-4c554a465e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-e3c46f9d-fc72-48a3-8c18-4c554a465e17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:13.881870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:13.882802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:13.883736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 277/552] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:13.893962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671b740-cd35-46fb-9b24-0df53d90e13b None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.893962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671b740-cd35-46fb-9b24-0df53d90e13b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.893962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671b740-cd35-46fb-9b24-0df53d90e13b None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.893962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 278/553] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:13.894666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:13.895439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:13.895439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:13.912336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:13.913616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:13.929564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 276/554] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:13.930176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:13.931305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:13.990625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 279/555] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:13.990625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.045935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.069726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb07b6e-4a74-43fe-a18e-f535f9076061 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.096027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d471e-ee50-40b6-8db6-e271c78bdd8c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.107791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.108896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.109404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.109871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.111160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:14.111773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.113471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.128440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 277/556] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:14.129670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.132468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.133469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.133997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.134480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.145579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.147240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:14.165838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:14.166757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:14.167645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:14.167645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:14.167645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:14.167645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:14.169759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 280/557] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:14.170728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.175369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.246547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fc9a0-ebbc-4ab1-a606-16a7139cdfb7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.277603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd528c9-ee89-4573-94fe-850910833303 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.283111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.284111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.284461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.284831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.296469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:14.305139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:14.306617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:14.307437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:14.311630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 278/558] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:14.312561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.317147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.320678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.321941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.322523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.322977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.335221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.337064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:14.354978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:14.355930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:14.356911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:14.356911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:14.356911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:14.356911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:14.359147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 281/559] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:14.360109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.420156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.431632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f84dae-1ab2-4302-b8fc-8ec88effe1fb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.468281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.468987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.469251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.469521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.470897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:14.471086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f97fd5-f6d6-41d3-8525-c207eb22a16d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.471204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.472573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.489253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 279/560] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:14.489253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.512207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.512911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.513194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.513457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.514289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:14.514525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.516081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a7c4b-1841-403d-b6b7-4fa75dd84211 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.529383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 282/561] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:14.529383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efafa5-cde0-440e-8e9a-d571fb4f6592 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.529383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efafa5-cde0-440e-8e9a-d571fb4f6592 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.529383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efafa5-cde0-440e-8e9a-d571fb4f6592 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.529383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 283/562] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:14.529383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efafa5-cde0-440e-8e9a-d571fb4f6592 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.555525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.598442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d438fb3-6cfd-43d8-95fa-2bf0a3d4ea4e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.634610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.635281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.635565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.635838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.636569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:14.636905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.638308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.639916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efafa5-cde0-440e-8e9a-d571fb4f6592 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.649888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 280/563] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:14.650715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.661173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.680375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.681224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.681506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.681946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.689534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.691004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-082c39b8-b34e-43d9-a28a-b9b6051027ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:14.705984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:14.706929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:14.707874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 284/564] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:14.708370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.709082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.709082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.725420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:14.726654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:14.768110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77209b4-0370-48e0-a4ec-8795b2acab1b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.792846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.800672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 285/565] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:14.800672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.814724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.815550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.815898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.816242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.829850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.831319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:14.849971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:14.851094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:14.852037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:14.852037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:14.852037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:14.852037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:14.852037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 281/566] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:14.852037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.910512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795ff38-8104-4824-ad30-d5fbe97c2a42 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:14.917826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:14.945441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:14.946244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:14.946523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:14.946794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:14.954040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:14.955566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:14.969983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:14.970978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:14.971917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 286/567] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:14.972521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:14.978350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8f22b-6e92-42cb-96a6-59af9f1ddad6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.014479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.015144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.015438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.015707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.016486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:15.016919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.018865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.031184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 282/568] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:15.031984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.064698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.076266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315054b-2630-4638-b57e-b07bd690ce3e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.109145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b64214-e6d0-48af-b882-281f56040ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.109974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64214-e6d0-48af-b882-281f56040ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.110258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64214-e6d0-48af-b882-281f56040ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.110573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64214-e6d0-48af-b882-281f56040ef9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.120759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b64214-e6d0-48af-b882-281f56040ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-01b64214-e6d0-48af-b882-281f56040ef9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:15.128619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:15.129543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:15.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 287/569] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:15.131593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced67325-9702-4b28-9932-5c9b65a0b335 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.131669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f53ebe-dd14-47d4-8ecb-33b3b9ad1ef5 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.131669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f53ebe-dd14-47d4-8ecb-33b3b9ad1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.131669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f53ebe-dd14-47d4-8ecb-33b3b9ad1ef5 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.132938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 288/570] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:15.133625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f53ebe-dd14-47d4-8ecb-33b3b9ad1ef5 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.161685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.178105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.179008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.179132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.179321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.180051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:15.180365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.181789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.193203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 283/571] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:15.194344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.238597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f53ebe-dd14-47d4-8ecb-33b3b9ad1ef5 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.272092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.272799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.273183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.273682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.275118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:15.275515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.276981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.286425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.294603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8472-6ffd-465a-8c54-d77a7d9df43c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.295403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 289/572] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:15.296224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.331230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.331809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.332092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.332363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.333231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=b96cbdec0f56412fb62617f03e77e713) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:15.333539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.334993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d359da-67eb-4719-99e8-75eb189c1f8a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.340821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 284/573] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/role_assignments?user.id=b96cbdec0f56412fb62617f03e77e713&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:15.341974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.342284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.342563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.361382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:15.363492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:15.395013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.412513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad09a4-dbc8-4a21-af88-fe381827de6f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.448592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 285/574] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:15.449653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.450346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.450627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.450898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.451659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:15.451964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.453426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.453827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.467730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 290/575] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:15.468937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.557160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.571609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cc942-6cef-4965-a748-f707b8bfdd6e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.578797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ff01-da3f-416a-a33a-147464a4149e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.608035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.608796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.609076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.609498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.614342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.615149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.615426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.615696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.616908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.618560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.626918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.628456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:15.633287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:15.634397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:20:15.635540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:15.637651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.639760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 286/576] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:15.640017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:15.649921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:15.651510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:15.654278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:15.654278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:15.654278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:15.654278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:15.654278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 291/577] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:15.654278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.758610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3428-6fd5-4391-b445-70b45d8f5a3c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.762820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113538ec-9699-4080-afe1-b9528d469532 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.793434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.794274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.794553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.794833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.802999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.803864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.804205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.804540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.806116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:15.813896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:15.814929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:15.816031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 292/578] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:15.816947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.821558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.823032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.833269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:15.845084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:15.846205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:15.847296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:15.847296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:15.847296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:15.847296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:15.853976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 287/579] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:15.854691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:15.880876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:15.922609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387b10-e320-4167-b54c-12ab323a3720 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.959698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:15.960383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:15.960690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:15.960971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:15.961821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:15.962119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:15.963557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:15.967479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517dd0f-0ea2-4bf4-b9f1-8ffa5b9404b0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:15.982731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 293/580] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:15.983570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.006190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.007409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.007804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.008170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.009044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.009450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.011037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.023731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 288/581] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.024563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.088788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.096296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a3d40-c06e-4bb2-bd14-0bdb9224d80a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.114314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.134038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3131a-ddfc-46db-8889-5be3cf8834b8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.134897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.135591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.135876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.136147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.136938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.137247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.138629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.147003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 294/582] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.147837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.175105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.175783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.176066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.176341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.177111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.177408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.178791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.191748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 289/583] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.192275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.266271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b1c47-6930-4434-91d9-d120780e7474 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.295380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2336a46-23c0-4dea-be90-d7d3e15d8b14 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.302153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.303619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.304114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.304562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.316217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.317116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.318759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.332072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.332900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.333175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.333445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:16.338596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:16.339484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:16.340379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:16.340379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:16.340379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:16.340379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:16.340601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.341597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.342477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 295/584] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:16.343479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:16.359536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:16.360724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:16.361983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:16.364151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 290/585] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:16.365205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.382089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.461823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155bfbf-060c-4344-8c76-c11266c2cd55 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.482881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3cec-0dea-4cd0-8042-79db91a61ccb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.499719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.500661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.500950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.501546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.515263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.516892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.518735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.519542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.519795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.520059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.527577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.529025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:16.535848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:16.536993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:16.537915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:16.537915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:16.537915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:16.537915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:16.540948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 296/586] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:16.542052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:16.543079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:16.543969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 291/587] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:16.545543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.594205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.643260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.656423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c1827-762f-44db-b28d-254e0cdd24ee None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.664639 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714df09-5f61-4568-a9c4-1828c8e7b654 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.693832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.694936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.695243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.695515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.696305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.696611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.698376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.699773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.700450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.700741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.701018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.701736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.702044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.703403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.712118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 292/588] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.713238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.720590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 297/589] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.721803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.820218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.824118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182dc6-9ef1-44c1-a623-17eab21b633d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.827257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a541363-242b-46b0-ac89-f8b71635740d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.860254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.861531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.861811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.862081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.862806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.863112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.864377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:16.864493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.865084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:16.865365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:16.865643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:16.866467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:16.867078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:16.868186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:16.881238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 293/590] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.881238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.888324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:16.897069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 298/591] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:16.897069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:16.991810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ae40e-d692-45aa-b27d-660f9b3562cf None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:16.997912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2175-3611-4964-9af8-0d132be7456f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.026490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.027402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.027681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.027957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0ca71f32f61e41288cac452e58b4a1dc/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.033581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.034671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.035133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.035582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.043000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.043132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=0ca71f32f61e41288cac452e58b4a1dc, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.044567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.044963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.058092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 294/592] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:16 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0ca71f32f61e41288cac452e58b4a1dc/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:17.058979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:17.059946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:17.060983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:17.062080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:17.064386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 299/593] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:17.065447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.088121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.140640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.200121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168a2af-9237-46c1-893c-6b849abc47c3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.215375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35dcf4-72f8-4cdf-a098-d58e7b89d3c0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.243012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.243753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.244033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.244303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.258315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.259779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.268672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.269738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.270019 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.270297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.278230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:17.279657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:17.280512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:17.281440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:17.281440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:17.281440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:17.281440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:17.281670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.283603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 300/594] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:17.284669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:17.295448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:17.296620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 295/595] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:17.297827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.383524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.399427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42decf-677c-4a38-a3e9-cb1b9cf6dc5f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.412866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.419792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e0853-3763-4188-a3e5-5fdcc0f83a45 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.436515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.437465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.437754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.438068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.438921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:17.439237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.440709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.459020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 301/596] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:17.459020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.461391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.461895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.462173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.462443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.463175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:17.463500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.464920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1fa36-c4ef-45e8-a24c-365f66e561b9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.477393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 296/597] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:17.478503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9385d-65f0-4831-bb6a-64c7da2b6692 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.478788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9385d-65f0-4831-bb6a-64c7da2b6692 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.479062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9385d-65f0-4831-bb6a-64c7da2b6692 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.480413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 297/598] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:17.484796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9385d-65f0-4831-bb6a-64c7da2b6692 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.567594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a04915-93a3-484f-89b0-63b03e40f226 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.591940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9385d-65f0-4831-bb6a-64c7da2b6692 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.605644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.606324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.606663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.607095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.607873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:17.608182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.609652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.622063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.633089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 302/599] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:17.633267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c933-97e4-47ea-96a3-91278d38f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.633408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.633501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c933-97e4-47ea-96a3-91278d38f624 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.633501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c933-97e4-47ea-96a3-91278d38f624 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.633501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c933-97e4-47ea-96a3-91278d38f624 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/a8e81a1c18d34adf8d8a45fa09f26099/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.649246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67c933-97e4-47ea-96a3-91278d38f624 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=a8e81a1c18d34adf8d8a45fa09f26099, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.651010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67c933-97e4-47ea-96a3-91278d38f624 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.672658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 298/600] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:17 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/a8e81a1c18d34adf8d8a45fa09f26099/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:17.672658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.672658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.672658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.678972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.686388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:17.687790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:17.766353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 299/601] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:17.767560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.777245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8d60d-e221-406c-9987-9a3ee3b9729f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.830250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.830454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.830747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.831290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.838972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.840453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:17.855472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:17.856424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 303/602] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:17.857987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:17.910490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee1ce4-6174-479d-a57d-d69ee50e7e62 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:17.923177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.957360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:17.958406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:17.958698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:17.959024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:17.965632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:17.975233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:17.977026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:17.978101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86b032-90ac-4b54-99b3-7bc812731378 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:18.002648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:18.003596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:18.004479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:18.004479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:18.004479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:18.004479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:18.011083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 300/603] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:18.012163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.020328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.021174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.021455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.021726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.036257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:18.047118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:18.048171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 304/604] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:18.053791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.146141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34896257-f6aa-4a3f-b65a-e435f89db05c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.176811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22578-d845-4ee5-a316-fb414acba8cd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.188079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.188793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.189071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.189680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.190138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:18.190465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.191903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.210497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 301/605] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:18.211796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.216555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.217612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.218090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.218555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.219581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:18.220243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.222610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.242973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:18.243696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 305/606] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:18.244917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.258464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:18.326665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6925a-5aba-4f3e-8259-d0487daed70b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.353888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d0647-a2f4-4e1c-b506-2c35cd50bc55 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.365098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.365923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.366149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.366429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.367260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:18.367480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.368884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.382962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 302/607] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:18.382962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.396441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.397728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.398200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.398679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/d61b8631879a461db96399a9b91682c0/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.414652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=d61b8631879a461db96399a9b91682c0, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.416891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.435525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 306/608] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:18 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/d61b8631879a461db96399a9b91682c0/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:18.437347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.475641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:18.492563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4bc12-62d8-47f0-a8e1-88658451eebb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.530006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.530006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.530006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.530006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.538463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.544831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.552295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:18.558816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:18.559831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:18.560948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:18.564227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 303/609] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:18.571038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.645753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0cbb2-9919-4c26-a0d9-09b46dae82e5 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.696991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.698203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.698679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.699145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.705606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b881a9-6837-4e8d-958c-715de4ab00cf None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.710529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.712219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:18.730675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:18.731615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:18.732504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:18.732504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:18.732504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:18.732504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:18.734794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 307/610] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:18.747833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.748557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.750030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.750310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.751218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:18.751606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.752022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.753610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.768651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 304/611] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:18.773158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:18.777609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:18.877960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce8e2-c33e-487f-b805-70e3672a7909 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.917128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:18.918057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:18.918387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:18.918658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/94b91e199622457787fa2a7e78d743c6/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:18.935201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=94b91e199622457787fa2a7e78d743c6, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:18.936763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:18.960314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 305/612] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:18 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/94b91e199622457787fa2a7e78d743c6/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:18.976103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f187c7-6eee-4c0f-a41d-c50d4173ad60 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:18.997134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:19.033516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:19.034220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.034508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.034790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.035535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:19.035848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:19.037298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:19.054982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 308/613] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:19.060576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:19.137570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:19.186618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65399e24-abd9-439e-8d38-123f797e82c4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:19.224335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:19.225122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.225401 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.225672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] PATH_INFO: `/v3/users/swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.233194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:19.234740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:19.249411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:19.250769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:19.256000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 306/614] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:19.257877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:19.408073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc994413-e36b-4af8-84aa-80d8f2107d90 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:19.454867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:19.455949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.455949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.456085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.456900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:19.457214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:19.458677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329fb54-c0bf-4211-8854-0aaeb6813793 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:19.486776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 309/615] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jun 30 12:20:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:19.492135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:19.510522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6ced8-0ba7-41dc-842a-1e23f0aec24b None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.510833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6ced8-0ba7-41dc-842a-1e23f0aec24b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.511082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6ced8-0ba7-41dc-842a-1e23f0aec24b None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.512742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 310/616] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:20:19.518161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.518445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.518721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.535249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:19.536454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:19.569382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:19.640798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 311/617] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:19.673961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:19.697894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69846a51-3b15-40ed-af43-0d4b091b58af None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:19.729673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:19.766697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:19.768703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.769261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.769712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.789443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:19.792164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:19.829541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:19.830550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:19.831460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:19.831460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:19.831460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:19.831460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:19.837628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 307/618] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:19.843707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:19.876658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ad095-870c-4d09-8565-a531fac28605 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:19.939885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cd179-1528-477f-8c28-b4dc1d432b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:19.940954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cd179-1528-477f-8c28-b4dc1d432b66 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.941344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cd179-1528-477f-8c28-b4dc1d432b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.941717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cd179-1528-477f-8c28-b4dc1d432b66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.954113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cd179-1528-477f-8c28-b4dc1d432b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2b7cd179-1528-477f-8c28-b4dc1d432b66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:19.962000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:19.962932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:19.963753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 312/619] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:19.968649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9ab9e-bc42-4977-b0f6-f32be88fb186 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:19.968649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9ab9e-bc42-4977-b0f6-f32be88fb186 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:19.968649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9ab9e-bc42-4977-b0f6-f32be88fb186 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:19.969029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 313/620] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:19.973574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9ab9e-bc42-4977-b0f6-f32be88fb186 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:19.997798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.007361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba91b9-c3c2-4f18-ad47-c716efa4056f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.052945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.053944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.055184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.055698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.057086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:20.057598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.059898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339afe7e-c3bd-4fa0-a8c4-b817b401e71d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.073519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 308/621] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:20.074889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c973039-5510-4157-818f-4c43848a4349 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.075182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c973039-5510-4157-818f-4c43848a4349 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.075461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c973039-5510-4157-818f-4c43848a4349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.085574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9ab9e-bc42-4977-b0f6-f32be88fb186 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.091710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5c973039-5510-4157-818f-4c43848a4349 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:20.092950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5c973039-5510-4157-818f-4c43848a4349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:20.112906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.125736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.127003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.127584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.128211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.129301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:20.129808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.131473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.141104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 314/622] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:20.142257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.168385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 309/623] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:20.168385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c973039-5510-4157-818f-4c43848a4349 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.242166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.264207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc5ed-880c-443f-8b3d-97701ae2b1e3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.287660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c973039-5510-4157-818f-4c43848a4349 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.304324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.305629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.306175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.306657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.314858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.316754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.323596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.324430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.324746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.325028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:20.335107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:20.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:20.337107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:20.337577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.338613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.340102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 315/624] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:20.341590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:20.357767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:20.358741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:20.359653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:20.359653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:20.359653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:20.359653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:20.359653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 310/625] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:20.360390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.371015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.463276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d88341-7aac-4125-a76c-096e2d9118cd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.468205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a81d2-a04a-49c2-b800-80e6f17fc34e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.507051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.507701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.507989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.508264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.509112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:20.509436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.510392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.511006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.511700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.512195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.512790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.524067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 311/626] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:20.525027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.526472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:20.535594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:20.536649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:20.537677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:20.540071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 316/627] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:20.541031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.544265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.597330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.630291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af693f8e-b5a8-4a80-a5c1-cdb118c678cb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.654189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fcfcd-fd5a-48fc-8f97-1d0e3bc6a548 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.665235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.666043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.666445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.666820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.667677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:20.668083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.669623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.682525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 312/628] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:20.683526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.694643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.695889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.696335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.696819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/b96cbdec0f56412fb62617f03e77e713/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.713219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=b96cbdec0f56412fb62617f03e77e713, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.715019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.731297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 317/629] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:20 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/b96cbdec0f56412fb62617f03e77e713/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:20.732658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.793396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade0716-4295-47b4-ab8d-e736239d33f0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.797766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.823778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:20.849671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.850395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.850756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.851097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.852539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:20.852539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.853846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.867755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9cb93-eeed-4336-91c8-cc2acc08f8d1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:20.868326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 313/630] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:20.873793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:20.925840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:20.927286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:20.928786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:20.929981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:20.947480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:20.951004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:20.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:20.985686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:20.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:20.996235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 318/631] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:20.999619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.062422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffd47-e3dc-47eb-b679-452d2e06412d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.101042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.102231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.102717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.103192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.118209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.120713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.137075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ed27d-2ff9-425d-bf1e-98373e63c6b4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-6b06745c-e1bb-4593-a27c-0caa9fb692ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:21.143841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:21.144825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:21.145726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:21.145726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:21.145726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:21.145726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:21.148708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 314/632] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:21.150280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bc3cb-61b4-46d3-bbd3-d31db1017a09 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.150773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bc3cb-61b4-46d3-bbd3-d31db1017a09 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.153450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bc3cb-61b4-46d3-bbd3-d31db1017a09 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.159835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 315/633] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:21.159835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bc3cb-61b4-46d3-bbd3-d31db1017a09 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.162744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.192451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.193517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.194821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.195296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.196664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:21.197327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.199636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c57d95-30d0-4609-920f-b705d2f92e72 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.224406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 319/634] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:20:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:21.226712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.227208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.227909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.249618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:21.251201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:21.253140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.299675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bc3cb-61b4-46d3-bbd3-d31db1017a09 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.348688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.349239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.349610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.349972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.350855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:21.351474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 320/635] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:21.351627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.352354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.353208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.376363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 316/636] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:21.378212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.469010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.474106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e382944-305a-41d0-9872-06f1fa63d503 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.494558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312b65-b8a1-4aa9-a1a7-c3280d49e790 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.496190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.512202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd67f-fce2-47e1-b328-ee2ad07ce115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.512887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd67f-fce2-47e1-b328-ee2ad07ce115 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.513169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd67f-fce2-47e1-b328-ee2ad07ce115 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.513444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd67f-fce2-47e1-b328-ee2ad07ce115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.531719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.532570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.532868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.533138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.543787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:21.551717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:21.552969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 317/637] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:21.558853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.592869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd67f-fce2-47e1-b328-ee2ad07ce115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.594353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd67f-fce2-47e1-b328-ee2ad07ce115 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.672126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d4709-fd8d-4de0-8aa9-2c8a18793527 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.698384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.706797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 321/638] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:21.708122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.708366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546b54d-bf08-41ee-a0f1-c0d69a4fb878 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.708472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546b54d-bf08-41ee-a0f1-c0d69a4fb878 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.708707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546b54d-bf08-41ee-a0f1-c0d69a4fb878 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.709568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.710003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 322/639] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:21.710130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.710520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.710681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546b54d-bf08-41ee-a0f1-c0d69a4fb878 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.721931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.723501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:21.738557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:21.739663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:21.740742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 318/640] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:21.745308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.770479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.830096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546b54d-bf08-41ee-a0f1-c0d69a4fb878 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.851960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cecc9-f08b-4184-b5f9-c7c21720a41c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:21.865248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.866043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.866419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.866835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.867586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:21.867941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.869363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf84c-dc7a-4519-b627-df50a118faa4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.879106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 323/641] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:21.880277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.880566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.880877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.894278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:21.895100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:21.895393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:21.895647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:21.901506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:21.902708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:21.908858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:21.910437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:21.929991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:21.930973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:21.931882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:21.931882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:21.931882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:21.931882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:21.931882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 319/642] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:21.932169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:21.970249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:21.981788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 324/643] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:21.993612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.016661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.048522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba1fc-5939-4992-8bde-3f6524127c3a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.086269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.087239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.088023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.088296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.089432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:22.089747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.091886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.105972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c8ac-e012-4641-8cca-411155de6af0 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.114307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 320/644] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:22.117511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.143760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.144572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.144885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.145158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.156016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.157556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:22.175745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:22.176644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:22.177706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:22.177706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:22.177706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:22.177706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:22.177706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 325/645] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:22.177706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.227995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.242188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487aa6b-4d6c-459a-9e2c-4f46304e8e80 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.274139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.280593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.281537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.282214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.282646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.283680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:22.283992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.284889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b2487-bc35-45dc-9789-314f57fad186 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.285403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.305395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 321/646] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:22.305395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.320337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.321170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.321447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.321767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.332610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:22.340334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:22.341341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:22.342177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 326/647] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:22.342874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.417488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0daec5-e324-4f70-bd8c-b391edd02b59 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.446879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16ed4-6934-470c-aa1b-afdee4556a70 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.449624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.455602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.456275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.456955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.457239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.483387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.483904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.484195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.484470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.485323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:22.485614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.487011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.499571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 327/648] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:22.500396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.524348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.534168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.535570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.616733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68372a02-3ece-4a01-b315-91e0ad684213 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.640011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 322/649] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:22.640901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.656841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.658035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.658310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.658606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.665487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.666952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:22.680783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:22.681757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:22.682666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:22.686806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 328/650] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:22.687574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.716451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.738493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.748364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09d836-7b67-4726-85d0-601d975c1e77 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.788502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.789205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.789493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.789950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.790597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:22.790823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.792219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.798999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883c3e-71ac-4f07-9086-8e1e94ab53a5 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.805178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 323/651] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:22.805178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.839720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.840529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.840840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.841114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.849194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.850728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:22.865758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:22.866716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:22.867643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 329/652] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:22.868233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.912527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc97d1-1c8c-42d7-9a94-0e27f27a5109 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.949961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:22.950883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:22.951258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:22.951532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:22.954171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:22.965296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:22.966746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:22.970619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653bab2-b534-4a12-85d9-123785b1eedb None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:22.985880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:22.986794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:22.987676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:22.987676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:22.987676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:22.987676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:22.987676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 324/653] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:22.988395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:22.997413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.010443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.011127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.011407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.011683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.012471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:23.012817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.014258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.027013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 330/654] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:23.027810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.088093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796999-cae5-4e7d-a2c8-30d27a44e0dd None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.122138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.122978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.123262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.123539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.127509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30725552-7b07-40cc-b9a0-25df26062162 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.134291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.135753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-92e1b6aa-e164-4acf-b8ba-abf7b4708f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:23.153009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:23.154089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:23.155158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:23.155158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:23.155158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:23.155158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:23.155158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 325/655] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:23.155511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9e055-e8de-47af-bdfd-3b4c7f600634 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.155748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9e055-e8de-47af-bdfd-3b4c7f600634 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.156024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9e055-e8de-47af-bdfd-3b4c7f600634 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.157328 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 326/656] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:23.158020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9e055-e8de-47af-bdfd-3b4c7f600634 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.166582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.167257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.167536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.167804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.168675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:23.168988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.170456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.187894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 331/657] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:23.188879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.191446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.220332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.262073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9e055-e8de-47af-bdfd-3b4c7f600634 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.286403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b788a5-7e8f-497e-a083-3271d630c04a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.296776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.297444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.297722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.297993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.298786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:23.299091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.300492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.313165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 327/658] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:23.313821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.324009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.324707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.325015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.325286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.326116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0ca71f32f61e41288cac452e58b4a1dc) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:23.326433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.327864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5de4bc-e751-46f5-9858-2f674660f90c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.334367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 332/659] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/role_assignments?user.id=0ca71f32f61e41288cac452e58b4a1dc&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:23.335631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.335920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.336198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.355811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:23.357094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:23.425214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae79cd-34a8-4e3a-babd-1efb24986324 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.428939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.444974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 333/660] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:23.445809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.447635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.469421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.470254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.470652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.470930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.471691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:23.472008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.473488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.485804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 328/661] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:23.486898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.574903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41ed29-479f-426f-aea2-1c772498e26c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.601952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d1333-72e4-4ce4-b3d0-201c46bca0a2 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.609990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.610856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.611174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.611656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.637144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.637996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.638281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.638554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.645470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.646923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:23.662892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:23.663886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 329/662] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:23.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.686324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.695484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.696980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.718987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.784371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24587c07-de3e-46ab-8b52-984115d9aae6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.808867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 334/663] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:23.809740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.827328 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.828418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.828907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.829424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.840084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:23.848012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:23.849240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:23.850308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:23.851388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 330/664] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:23.853892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.914177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.921849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952eb6e-9083-4f4e-a387-1b951fa5ae83 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.953260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:23.959192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40899a89-fa21-417d-83f1-19f82cc8cad6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:23.960730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:23.961544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:23.961794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:23.962059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:23.969285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:23.970799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:23.991794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:23.993157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:23.994145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 335/665] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:23.994768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:23.999605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.000522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.000832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.001109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.002541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:24.002847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.004300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.023574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 331/666] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:24.023574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.105609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b1125-7575-4024-b58c-f68aca6f7143 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.127764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff82db-bf4e-4afe-9281-67908d6ca4c0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.141848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.142880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.143177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.143449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.154381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.155829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.162842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.163979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.164267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.164542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.168041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:24.168041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.168041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:24.174068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:24.174953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:24.175847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:24.175847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:24.175847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:24.175847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:24.175847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 336/667] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:24.176127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.184742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 332/668] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:24.185834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.191561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:24.210706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:24.286389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f6a0-8a0b-49c7-8bfb-2adbf9cd359c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.291926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a4be3-5323-4742-aef8-52d6c4223563 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.322881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.323577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.323859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.324131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.324998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:24.325310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.326766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.327262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.328357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.328862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.329359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.330880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=a8e81a1c18d34adf8d8a45fa09f26099) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:24.331418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.333461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.349045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 333/669] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/role_assignments?user.id=a8e81a1c18d34adf8d8a45fa09f26099&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:24.349045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.354204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 337/670] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:24.355214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.434029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:24.458423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:24.476391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e4344-61cb-43ef-8d17-57d576056c14 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.482759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84053ca7-d825-4d72-a572-83ad776f16fb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.520544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.522901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.522901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.522901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.535284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.536143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.537619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.537747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.537823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.537996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.539275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:24.539275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.541117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.558387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 334/671] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:24.559287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:24.561685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:24.562785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:24.563880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:24.563880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:24.563880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:24.563880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:24.566108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 338/672] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:24.566108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.687219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973cfd4-eb25-4c6e-ab14-c87a4cbb5ac7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.701615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b792-687e-4cb9-9a40-d8df4b28a743 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.724820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.725622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.725906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:24.726069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.726795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.743596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:24.748709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.750625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.751081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.751502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.761273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.763466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:24.782149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:24.783547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:24.784662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:24.787411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 339/673] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:24.793461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.810492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.812121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.905041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8830d0-a58e-4004-a17d-461878c51f63 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:24.939228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 335/674] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:24.941111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.951037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:24.951952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:24.951952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:24.952244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:24.953012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:24.953319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:24.954953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:24.967383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 340/675] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:24.968402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:24.998034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.018080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.064924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233829-5fab-4b70-8b70-58ceac593597 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.095957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8228c51-5049-4c85-9404-a6c127cef2e6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.107395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.108711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.109145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.109621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.111214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:25.111699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.113892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.129458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 336/676] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:25.130526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.139862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.140957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.141787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.142313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.153883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.155638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:25.175095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:25.175980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:25.176903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:25.176903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:25.176903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:25.176903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:25.178852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 341/677] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:25.179808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.243958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9867f-8d24-4d10-9ae5-d0e2b18fe986 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.246371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.278514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.281134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.281948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.282229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.282679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.290421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cee433-9086-4044-bbd6-419483f4ebf3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.291859 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.293665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:25.308708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:25.309892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:25.311057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:25.314705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 337/678] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:25.315721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.330644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.331566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.331993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.332413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.333585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=d61b8631879a461db96399a9b91682c0) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:25.334069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.335668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.342676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 342/679] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/role_assignments?user.id=d61b8631879a461db96399a9b91682c0&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:25.343731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.499433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a62efa-b27e-4654-b844-bc70150ff3f8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.524389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925fb33-d899-4283-a052-35dd3424ebbc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.545009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.545694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.545979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.546254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.546984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:25.547295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.548790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddf8d0-2a7b-49ff-a271-1d9cab495e1a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.563154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 338/680] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:25.564033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41081ae6-967c-4bbd-82c5-f2eef245f7e6 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.564320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41081ae6-967c-4bbd-82c5-f2eef245f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.564593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41081ae6-967c-4bbd-82c5-f2eef245f7e6 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.570718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 339/681] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:25.571482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41081ae6-967c-4bbd-82c5-f2eef245f7e6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.572080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.573101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.573609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.574088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.575130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:25.575635 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.577349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32408178-6162-451d-90ba-fd150f2152d5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.599465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 343/682] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:25.600609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.600921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.601190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.602774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.620275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:25.621544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:25.630357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.690403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41081ae6-967c-4bbd-82c5-f2eef245f7e6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.726173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 344/683] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:25.726173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.735696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.736563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.736883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.737156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.750461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.751925 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:25.766801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:25.767729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:25.768619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 340/684] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:25.773866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.861684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.862614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba8a72-7e75-4692-be8c-728379fa7208 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.879348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1958e-8672-48c6-91df-5dfb3c8df2bc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:25.907056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.908162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.908815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.909603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.914033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:25.915172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:25.915172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:25.915375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:25.925562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:25.926755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:25.934601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:25.935562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:25.936395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:25.941150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 341/685] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:25.942062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:25.990603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:25.992129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.071335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967228-6d58-4c83-bcf3-8d7f518a64f1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.098537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 345/686] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:26.098537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.117272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.118138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.118520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.118918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.119904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:26.120336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.121990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.139808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 342/687] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:26.140802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.144675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.150015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.222475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04beace6-ef91-4a9b-9339-2573763916d5 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.256760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9460-64a2-4938-b3aa-1102d1594641 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.264004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.264819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.265131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.265412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.266215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:26.266523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.268081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.281007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 346/688] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:26.282060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.298219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.299191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.299839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.300131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.313410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.314990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:26.334619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:26.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:26.337031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:26.337031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:26.337031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:26.337031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:26.337031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 343/689] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:26.337031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.398939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d15f2-51dc-4167-ac25-6513aedc8a23 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.403043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.413284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.441314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.442162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.442540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.442909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.444343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=94b91e199622457787fa2a7e78d743c6) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:26.444914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.446470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.457013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 347/690] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/role_assignments?user.id=94b91e199622457787fa2a7e78d743c6&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:26.458132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.464404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afcde3-efb8-47dd-a0b6-93bc910a92a5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.507011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.508447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.508784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.509065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.521813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.523856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:26.545050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:26.547224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:26.549621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:26.549621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:26.549621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:26.549621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:26.549621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 344/691] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:26.555536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.635094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c672c0-d49f-43f8-b42f-cfdf27cc4cd1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.674253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.675294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.675748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.676235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.677313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:26.677811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.679387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0acb7-c1c7-43e4-a68c-a4063a67916a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.679612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6956e-73b0-4c25-9d40-c26e596c2d4e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.690591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.693227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 348/692] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:26.694823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7365f-266b-4ed2-9487-d29f4aa084c9 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.696978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7365f-266b-4ed2-9487-d29f4aa084c9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.696978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7365f-266b-4ed2-9487-d29f4aa084c9 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.696978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 349/693] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:26.701857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7365f-266b-4ed2-9487-d29f4aa084c9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.723907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.725060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.725948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.727196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.727196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:26.727462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.729024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0070af-8251-49d7-a91e-7e338394930c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.733441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.744567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 345/694] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:26.745927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.746215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.746491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.769782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:26.771257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:26.831330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7365f-266b-4ed2-9487-d29f4aa084c9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:26.860676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 346/695] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:26.860676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.877884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:26.879050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:26.879504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:26.879958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:26.897922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:26.899669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:26.917263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:26.918242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:26.919142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:26.921722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 350/696] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:26.922848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:26.970203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:26.974779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117abe0-2aff-4042-9e27-2aa8cf0a1ad2 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.011570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.012798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.013257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.013701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.021739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.021993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.023723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.036102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528b72c-2c47-4f98-a112-3c9aa4fe1a91 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:27.039955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:27.040927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:27.041875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:27.044651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 347/697] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:27.045808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.083531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.084346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.084639 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.085006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.097144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.099371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:27.118333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:27.119283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:27.120159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:27.120159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:27.120159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:27.120159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:27.124420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 351/698] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:27.125346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.173321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef8a4e-d587-4ca6-aef5-ddc53f89e9de None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.217488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.218168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.218452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.218727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.219525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:27.219866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.221924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.236315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 348/699] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:27.237299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.241994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10307b13-1297-41b3-b76c-54425c7ffbb7 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.256358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.281183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.282490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.282973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.283443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.284583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:27.285159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.286966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.297773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.308310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 352/700] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:27.309762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.349234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f0330-bc66-4327-ae65-e1b7efcb2cb3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.385397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.386165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.386452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.386728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.387466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:27.387771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.389185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.407215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 349/701] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:27.408686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.427878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90a663-d1da-4cc8-90c3-dc39750c539e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.464801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.465803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.466279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.466745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.467986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=b96cbdec0f56412fb62617f03e77e713) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:27.468554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.470535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.477894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 353/702] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/role_assignments?user.id=b96cbdec0f56412fb62617f03e77e713&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:27.479094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.493653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.518282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d1d3a-b70e-4ed3-a914-68a1a13cde2a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.556994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.558674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.559364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.559648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.560082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.604023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c7646-95ae-4300-b65c-0e2906875809 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.639114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.640729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.644961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3a78-356f-4f4c-91b4-656236f969dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.646006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3a78-356f-4f4c-91b4-656236f969dd None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.646501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3a78-356f-4f4c-91b4-656236f969dd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.646981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3a78-356f-4f4c-91b4-656236f969dd None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.647886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a3a78-356f-4f4c-91b4-656236f969dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:27.649679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a3a78-356f-4f4c-91b4-656236f969dd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:27.673915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 354/703] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:20:27 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:27.675706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52757f-0087-4dd0-b715-13050a62e12e None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.676239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52757f-0087-4dd0-b715-13050a62e12e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.676734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52757f-0087-4dd0-b715-13050a62e12e None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.682623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 355/704] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:27.685692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.685692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.685692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:27.708425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:27.709894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:27.771610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.808588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 350/705] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:27.815508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.817527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 356/706] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:27.848480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:27.917789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:27.954291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e2328-d2f8-4b29-b85d-859a324570e0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.987649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce902a-47af-4d5c-a4e7-e9c42202b458 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:27.990291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:27.991264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:27.991688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:27.992124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.003407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.005135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.023502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:28.023860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:28.024907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:28.025954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:28.025954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:28.025954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:28.025954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:28.026184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.026184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.026184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.026184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.027008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.028193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 351/707] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:28.036349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:28.051185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:28.052387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:28.053492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:28.056854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 357/708] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Jun 30 12:20:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:28.064692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:28.088930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:28.184241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06e12-01d3-4d1e-8e15-65ffc8f8b7f6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:28.210116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72782780-4568-44a9-b712-1a533cb30cbc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:28.227190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:28.232914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.233765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.234226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.234696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.235699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:28.236182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.237846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.250375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 352/709] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:28.260456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:28.261762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.262450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.262730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.263002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.263718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:28.264029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.265469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.280658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 358/710] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jun 30 12:20:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:28.284576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:28.399594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:28.411220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d71a2-329f-4838-8368-bc4723c317ed None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:28.453073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.455206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.455653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.456122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.464192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.465879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:28.483255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:28.484376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:28.485311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:28.488015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 353/711] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:28.494049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:28.546136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ebcac-97ea-4f00-a9ee-f48fab289039 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:28.582568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.583446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.583869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.584238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.585400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:28.585400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.586951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.593451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 359/712] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jun 30 12:20:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:28.640776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753b3dc-2cd1-4ce5-ba9b-a1986c0edce0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:28.702934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.703637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.703916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.704193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.705216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:28.705527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.707029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffe4fa-1a53-49cf-b2b7-3655f5699b77 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.726041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 354/713] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:28.732063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:28.796815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:28.859401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100964-fd37-4773-b467-eb4d06da4f27 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:28.880445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974bb0f-c54a-4db2-b344-c53724a3bd85 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.880817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974bb0f-c54a-4db2-b344-c53724a3bd85 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.881024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974bb0f-c54a-4db2-b344-c53724a3bd85 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.882640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 355/714] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:28.887965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db827363-76d3-43ae-8c4b-141be340166e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.888254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db827363-76d3-43ae-8c4b-141be340166e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.888559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db827363-76d3-43ae-8c4b-141be340166e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.906724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:28.907050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-db827363-76d3-43ae-8c4b-141be340166e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:28.907999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:28.908376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-db827363-76d3-43ae-8c4b-141be340166e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:28.908753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:28.909225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:28.910280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0ca71f32f61e41288cac452e58b4a1dc) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:28.910779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:28.911497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:28.912438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:28.925009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 360/715] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:28 2022] GET /identity/v3/role_assignments?user.id=0ca71f32f61e41288cac452e58b4a1dc&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:28.988058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 356/716] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:29.012362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:29.126945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628bec1-6695-451f-9e2f-21d90da220fc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:29.129820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:29.166389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:29.167645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:29.168120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:29.168587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:29.169508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:29.171149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:29.183987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:29.185176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:29.186070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 361/717] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:29 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:29.191945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db827363-76d3-43ae-8c4b-141be340166e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:29.315495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db827363-76d3-43ae-8c4b-141be340166e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:29.361494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:29.361900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:29.362181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:29.362452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:29.363178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:29.363544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:29.364971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0a2f-9982-4593-9a51-ce8841b696c9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:29.370095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 357/718] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:29.375475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:29.507270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe26a-5b8d-491a-8f48-0341ec9896a8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:29.542316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:29.542989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:29.543267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:29.543535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:29.544239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:29.544540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:29.545946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4c868-67e8-4cc1-8079-c594eefb34d7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:29.550998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 362/719] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:30.253853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcdaa9-9e09-404a-a814-5c730f8dc606 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.254415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcdaa9-9e09-404a-a814-5c730f8dc606 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.254415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcdaa9-9e09-404a-a814-5c730f8dc606 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.256079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 358/720] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:30.261738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.262140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.262332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.283378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a30bbe-992c-4c35-847c-91d2a805d30a None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.283690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a30bbe-992c-4c35-847c-91d2a805d30a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.283927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a30bbe-992c-4c35-847c-91d2a805d30a None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.285332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 359/721] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:20:30.286882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:30.288085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:30.299072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.299361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.299635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.313781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:30.316922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:30.331510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:30.332748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:30.386030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 363/722] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:20:30.417485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:30.460020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 360/723] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:30.479745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:30.581317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc7a43-62a4-4736-9568-c4ab87864c20 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:30.644237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70cf50-af7d-48b5-a583-39148354f13e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:30.647154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:30.647968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.648243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.648510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.674888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:30.687226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:30.688235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:30.689248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:30.689897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 364/724] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:30.701624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:30.718150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:30.719296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.719744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.720257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.726711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:30.731633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:30.750372 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:30.758828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:30.759987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:30.761135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:30.769096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 361/725] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:30.775003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:30.824267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f9f7b-3e86-44db-b32b-6fe74cb14c03 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:30.906865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74880690-77d5-49eb-bf80-45e0b93744b1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:30.917328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:30.918023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.918308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.918579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.919334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:30.919640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:30.921068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:30.940093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 365/726] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:30.942232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:30.942908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:30.943185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:30.943458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:30.944579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:30.945176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:30.946082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:30.946368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:30.956311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 362/727] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:30.961681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.025166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.070589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253c037-d4c3-45de-a544-accfd3edf74c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.105275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.106085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.106365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.106637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.119305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.120983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.137473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc358e-b67a-45d5-ac6b-5f5644fb4c8e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.141046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-12d5923e-a4b4-4a94-b161-c65ce9f3de1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:31.150294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:31.151253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:31.152681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:31.152681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:31.152681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:31.152681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:31.152681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 363/728] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:31.153465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6343-e4e5-4772-8c99-ffcd2eb8b4ee None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.153828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6343-e4e5-4772-8c99-ffcd2eb8b4ee None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.154138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6343-e4e5-4772-8c99-ffcd2eb8b4ee None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.156897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 364/729] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:31.161646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6343-e4e5-4772-8c99-ffcd2eb8b4ee None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.200074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.201339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.201622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.202153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.215777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.217465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-1343c851-d623-41e0-b1c1-76ab1557e0d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:31.239098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:31.240045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:31.241331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:31.241331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:31.241331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:31.241331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:31.241331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 366/730] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:31.242038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.242340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.242864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.264019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:31.265293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:31.302123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6343-e4e5-4772-8c99-ffcd2eb8b4ee None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.313613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.346444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.347486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.347919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.356804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 367/731] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:31.356804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.356993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.356993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:31.356993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.356993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.380199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 365/732] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:31.382921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.433973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.479662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45018ec7-0fa2-41e1-8ce9-473b91fb87d3 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.520743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e07c53-7785-4969-ad33-22b8bc3e49d1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.526805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.527730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.528705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.529187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.530086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:31.531047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.533123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.549901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 368/733] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:31.551306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.561832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.562257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.562544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.562820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.563594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.565264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.596657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 366/734] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:20:31 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:31.596657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.655739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.700216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a644b14-11d1-4c93-a751-ae13131d0712 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.728170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.744806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.745656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.745937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.746211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.751909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d948b07-0fc3-40fe-988e-c145b5c51a33 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.755089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.756571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-674af9c6-cf5d-455e-8054-1437a4bbab94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:31.772506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:31.773574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:31.774503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 369/735] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:20:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:31.775161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5769-755d-4fd8-8a0c-d8401acc83ab None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.775828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5769-755d-4fd8-8a0c-d8401acc83ab None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.775828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5769-755d-4fd8-8a0c-d8401acc83ab None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.777226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 370/736] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:31.782005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5769-755d-4fd8-8a0c-d8401acc83ab None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.805660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.806520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.806799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.807075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.817522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.818393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3b04e2d1-28aa-40be-9873-f5ed053ac149 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:31.834253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:31.835231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:31.836171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 367/737] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:31.841672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.841672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.841672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.858756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:31.860265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:31.915292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5769-755d-4fd8-8a0c-d8401acc83ab None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:31.948964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 368/738] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:31.950172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:31.964031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:31.964746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:31.965025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:31.965293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:31.966143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:31.966451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:31.967365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:31.968197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.001695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 371/739] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jun 30 12:20:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:32.002790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.054042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.067780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf9e1-6022-4211-8ac6-e81761fe8128 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.109134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.109824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.110220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.110557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.111409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:32.111718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.113187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.128335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124887-dcd3-4e1f-b597-760910962852 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.132678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 369/740] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jun 30 12:20:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:32.132678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.175209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a75d-6595-4372-90e9-da413c476c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.176103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a75d-6595-4372-90e9-da413c476c71 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.176451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a75d-6595-4372-90e9-da413c476c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.176860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a75d-6595-4372-90e9-da413c476c71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.188336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a75d-6595-4372-90e9-da413c476c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-a2c2a75d-6595-4372-90e9-da413c476c71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:32.196257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:32.197194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:32.198054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 372/741] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:32.198740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76347c07-e181-4f32-926a-032e131c8891 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.199015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76347c07-e181-4f32-926a-032e131c8891 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.199278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76347c07-e181-4f32-926a-032e131c8891 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.205004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 373/742] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:32.205004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76347c07-e181-4f32-926a-032e131c8891 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.215640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.250137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f15d6-13db-4e35-a3b3-a8ce4b7b1622 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.286364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.287021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.287296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.287562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.312908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.317274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76347c07-e181-4f32-926a-032e131c8891 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.352948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d460a99-38ba-4dc8-a47b-2247f5282a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.353523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d460a99-38ba-4dc8-a47b-2247f5282a9e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.353799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d460a99-38ba-4dc8-a47b-2247f5282a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.354063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d460a99-38ba-4dc8-a47b-2247f5282a9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.359878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.361483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.424552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d460a99-38ba-4dc8-a47b-2247f5282a9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.426067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d460a99-38ba-4dc8-a47b-2247f5282a9e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.434038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.466578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 370/743] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:32.467391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.539377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.547505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 374/744] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:20:32.547505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.547505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.547505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.563469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:32.564958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:32.576851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c25d5-1b33-4f1e-b610-4c9a06987a49 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.613351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.613949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.614221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.614491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.615228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:32.615537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.616950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.625400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 371/745] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:32.626241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.648657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 375/746] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:20:32.648657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.663070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.730192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b22eb-207d-4cb4-8284-332e06d81596 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.748836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72069-971b-48a1-a5f2-970d08141097 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.765505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.766431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.766747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.767089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.777667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.778773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.780226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.784396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.785240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.785525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.785798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.796542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:32.798064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:32.798952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:32.799904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:32.799904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:32.799904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:32.799904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:32.800134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.800203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 372/747] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:32.800794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:32.824208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:32.825429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:32.826841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:32.826841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:32.826841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:32.826841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:32.826841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 376/748] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:32.826841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.897781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:32.913091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba02f69-2786-4d1a-abf7-56c546137d63 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.941417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33356c0c-c867-4ebb-8ade-6d48218390b9 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:32.947892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.948726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.949015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.949286 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.960556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.962087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:32.978558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922ba60-9313-4735-96f7-7857700211df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:32.979234 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922ba60-9313-4735-96f7-7857700211df None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:32.979515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922ba60-9313-4735-96f7-7857700211df None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:32.979623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:32.980427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:32.986634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:32.986634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:32.986634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:32.986634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:32.986866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922ba60-9313-4735-96f7-7857700211df None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:32.986866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922ba60-9313-4735-96f7-7857700211df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:32.987007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 373/749] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:32.987007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:32.987138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922ba60-9313-4735-96f7-7857700211df None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.009062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 377/750] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:20:32 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:33.010100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922ba60-9313-4735-96f7-7857700211df None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.071840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.132090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85be73-f3d2-4de6-86f8-a55d941beea1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.162121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922ba60-9313-4735-96f7-7857700211df None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.179208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.179888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.180163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.180432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.181978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.182281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.184403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.193125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.202118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 374/751] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.202118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.214544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.215476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.215914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.216191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.216968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.217279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.218666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.236980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 378/752] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.236980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.340714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774deb0-414d-40a9-8189-ca2c51af90a4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.349123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.353644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49af561-96c0-4b0f-be4b-55ad9e6bbbe0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.377275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.378667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.379238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.379714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.380653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.381181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.382762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.391144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.391996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.392307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.392579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.397032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 375/753] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.398177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.404872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.406354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:33.422236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:33.423386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:33.424284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:33.428260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 379/754] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:33.429324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.463723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.515723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726263a-ae2d-4bf1-b12c-e35c54c7c8c8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.552540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95aac7-45e2-4d28-bb3c-0df530a2d988 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.562412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.563991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.564725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.565398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.576477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.579156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.592096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.592950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.593328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.593694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:33.599447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:33.600840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:33.602257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:33.603689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.604650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.611381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 376/755] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:33.612855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:33.623107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:33.624094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:33.625070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:33.626594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 380/756] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:33.626594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.653070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.725068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ad102-1250-4d4d-9faa-82c9921f8ebe None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.746375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c65c25-ead6-42e4-8cd8-4e1a801387ce None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.747718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.765538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.766207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.766568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.767194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.767988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.768297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.769795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.781529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 377/757] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.782763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.790766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.791451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.791722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.791990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.792817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.793121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.794554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e4b36-3288-4781-9d01-d2928fe21286 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.808334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 381/758] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.809649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb5dfc-5565-43dd-b0a6-823c634d27c3 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.809925 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb5dfc-5565-43dd-b0a6-823c634d27c3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.810208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb5dfc-5565-43dd-b0a6-823c634d27c3 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.812736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 382/759] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:33.812736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb5dfc-5565-43dd-b0a6-823c634d27c3 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.898438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e12d6c-45d3-42c9-bc6a-9f25c0657e84 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.899499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:33.924476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb5dfc-5565-43dd-b0a6-823c634d27c3 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:33.942774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.943458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.943733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.944347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.945307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.945602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.947034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.963935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:33.964604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.964914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.965187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:33.966015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=a4dec05937f84eedbbd5795582a16964) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:33.966318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:33.967766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a25ff-256b-4517-82c2-46591fc35a8c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:33.969580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 378/760] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.970726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:33.988934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 383/761] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/role_assignments?user.id=a4dec05937f84eedbbd5795582a16964&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:33.990452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:33.990746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:33.991022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:34.022743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:34.027016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:34.028571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:34.116687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3697e5-e22d-40ad-a002-50964bc1d936 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:34.117405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 384/762] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:34.118696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:34.157783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:34.158589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:34.158824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:34.159391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:34.160474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=b96cbdec0f56412fb62617f03e77e713) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:34.160823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:34.163025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:34.181172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 379/763] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:33 2022] GET /identity/v3/role_assignments?user.id=b96cbdec0f56412fb62617f03e77e713&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:34.181172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:34.286764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:34.353746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327549fb-2da1-4874-a256-7ffbed5e9735 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:34.390262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:34.394722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7526a4-869e-423e-b6f3-0f16e3e256d1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:34.437166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:34.438848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:34.439433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:34.440242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:34.440522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:34.440822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:34.441668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:34.441943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:34.442606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:34.444368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:34.463294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:34.464797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:34.468856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:34.469855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:34.471544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 380/764] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Jun 30 12:20:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:34.478067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:34.504420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:34.506534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:34.508652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:34.508652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:34.508652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:34.508652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:34.514192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 385/765] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:34.515048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:34.667509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e4c2d-0173-4fd9-87c9-667a79a529dc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:34.696479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:34.713175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:34.713855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:34.714138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:34.716193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a24c-2259-4b95-a356-3315a1389ebd None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:34.720903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:34.721690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:34.722003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:34.723409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:34.751927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 386/766] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:34.756973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:34.760990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:34.761686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:34.761960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:34.762230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:34.763833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:34.764141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:34.765566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:34.778859 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 381/767] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jun 30 12:20:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:34.785969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:34.833981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:34.930316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a1078-88e7-42f7-8703-9ad53022c9e9 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:34.967024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401c625-2d6d-48b1-a78a-2e1e8ab74fba None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.001842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.002835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.003121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.003399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.004116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:35.004426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.005851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.013467 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 382/768] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jun 30 12:20:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:35.019216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.025934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.026757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.027033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.027300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.045465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.046954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:35.084888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:35.085827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:35.086719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 387/769] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:35.092142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.117791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:35.127168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dcae-51ad-424d-ac01-00ab03574ef3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.163307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.164421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.164900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.165194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.166015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] RBAC: Adding query filter params (interface=public, service_id=ce530f81552b4b09bab3a98edd2acf88, region_id=RegionOne) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:35.166474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.167894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.177057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 383/770] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:20:35 2022] GET /identity/v3/endpoints?service_id=ce530f81552b4b09bab3a98edd2acf88&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:35.183211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.225039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:35.290715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7f06b-4e4b-4e43-849f-17d93cfe3369 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.296943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560d59c-09e9-47ff-adab-ae7db137be19 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.333201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cded6fd-aa69-48c9-a89f-b86847fdd258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.333999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded6fd-aa69-48c9-a89f-b86847fdd258 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.334345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded6fd-aa69-48c9-a89f-b86847fdd258 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.334684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded6fd-aa69-48c9-a89f-b86847fdd258 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.335442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cded6fd-aa69-48c9-a89f-b86847fdd258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.336954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cded6fd-aa69-48c9-a89f-b86847fdd258 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.337685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.339142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.339622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.340211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.341322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:35.341825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.348653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 384/771] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:20:35 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:35.348653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff146-b9b9-4154-919b-b13b83ff4723 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.348653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff146-b9b9-4154-919b-b13b83ff4723 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.348653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff146-b9b9-4154-919b-b13b83ff4723 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.349660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.350752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 385/772] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:35.354636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.354636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.354636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.384892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 388/773] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:35.386032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:35.387278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:35.390881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.417880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:35.481605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 386/774] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:35.505271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.578599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a9ff4-f059-453e-82ba-5f5c54a459ac None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.583351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:35.618665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.619576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.619853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.620123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0ca71f32f61e41288cac452e58b4a1dc/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.622155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=0ca71f32f61e41288cac452e58b4a1dc, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.623595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.631700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8e918-7ac4-4fe0-8811-60bf1b68164e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.644969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 389/775] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:35 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0ca71f32f61e41288cac452e58b4a1dc/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:20:35.682194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.683410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.683868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.684321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] PATH_INFO: `/v3/services/network` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.685225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.686912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:35.702455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:35.703527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:35.704490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:35.707996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 387/776] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:35 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:35.714435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.829425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2bdf4-3630-48b9-a034-3e9f5535cfe9 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:35.867301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:35.868250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:35.868731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:35.869140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:35.870007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:35.870451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:35.871971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:35.878230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 390/777] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:35.885840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:35.904964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:36.005664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419134b-c9e5-4a69-8706-af029304c35c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:36.043508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:36.044179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.044456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.044758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.045474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] RBAC: Adding query filter params (type=network) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:36.045770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:36.047140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:36.052981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 388/778] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:36.059010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:36.093940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:36.180678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165655-3361-4a6e-95c2-8d4ff71707e0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:36.217960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:36.218637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.218918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.219179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.220177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] RBAC: Adding query filter params (interface=public, service_id=9cd81ff317be4bd29cbb04c4093f8269, region_id=RegionOne) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:36.220536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:36.222028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63778d9c-39a2-4cb3-ab43-22af1cc3308e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:36.234469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 391/779] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:20:36 2022] GET /identity/v3/endpoints?service_id=9cd81ff317be4bd29cbb04c4093f8269&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:36.240345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:36.262193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d39e0-7c43-443f-934a-f1bb7bc65a49 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.262508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d39e0-7c43-443f-934a-f1bb7bc65a49 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.262765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d39e0-7c43-443f-934a-f1bb7bc65a49 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.264393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 392/780] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:36.269576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.269861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.270139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.293786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:36.295003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:36.376599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:36.407795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912c9f2-b7f4-4988-a373-0cba84394371 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:36.426149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 393/781] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:36.470574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7ab15-f0a8-40bf-8543-fdf554a513f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:36.471347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7ab15-f0a8-40bf-8543-fdf554a513f1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.471631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7ab15-f0a8-40bf-8543-fdf554a513f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.471906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7ab15-f0a8-40bf-8543-fdf554a513f1 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.472617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7ab15-f0a8-40bf-8543-fdf554a513f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:36.474030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7ab15-f0a8-40bf-8543-fdf554a513f1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:36.479548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:36.483232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 389/782] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:20:36 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:36.507469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:36.590749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b3ca4-6203-4e1a-b103-fe54920a2375 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:36.603491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23590764-cc07-496a-a27c-dfb28b29588f None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.603815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23590764-cc07-496a-a27c-dfb28b29588f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.604051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23590764-cc07-496a-a27c-dfb28b29588f None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.605804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 390/783] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:36.613121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.613439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.613684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.629402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:36.630170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.630448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.630719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.642872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:36.650790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:36.651795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:36.652887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 394/784] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:36.659788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:36.662769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:36.664024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:36.774741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:36.777160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 391/785] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 30 12:20:36.799709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777b260-23c9-44da-a436-1c6fb7f29b9f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:36.803237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:36.857404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:36.858087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.858369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.858640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.859399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:36.859705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:36.861330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:36.877060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 395/786] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:36.882641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:36.908917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:36.944405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c036700-4407-4a00-9c62-ffdbe675a925 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:36.981594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:36.982405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:36.982682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:36.983121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:36.995575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:37.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:37.005381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:37.006339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 392/787] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:37.015041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.083645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb9dab-fd06-47e3-9ec6-151fbdbbe36c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.130985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:37.131802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:37.132086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:37.132361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:37.157708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:37.159095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:37.169520 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:37.193011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:37.194331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:37.195210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:37.195210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:37.195210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:37.195210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:37.196130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 396/788] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:37.201253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb37d8-4c60-40d7-a7ad-3deeb93765ce None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.202781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.259229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:37.272070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:37.272882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:37.273161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:37.273439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:37.274298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:37.274610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:37.276025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:37.291659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 393/789] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:37.299125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.390365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8df62-4feb-406e-80cf-dc9d0f67d619 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.454794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:37.455571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:37.455730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:37.456001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:37.456756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:37.457068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:37.458446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:37.479352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be267ab-8f0d-4a46-8924-2f3e991880dc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.482019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 397/790] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:37.483050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.538975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:37.540200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:37.540820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:37.543196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:37.562624 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:37.564436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:37.587564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:37.596296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:37.597227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:37.598153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:37.598153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:37.598153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:37.598153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:37.600416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 394/791] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:37.606263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.671714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:37.675418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04bdb4-56ee-40f8-af75-49bf16310c6d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.767798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02322edb-aecb-43a1-95ce-39473e609a36 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.773345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:37.774187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:37.774467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:37.774739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:37.797709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:37.799270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:37.803073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:37.803752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:37.804031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:37.804308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:37.805067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:37.805374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:37.806758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:37.820439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 395/792] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:37.826293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:37.827230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:37.828190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:37.832235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 398/793] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:37.833424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.833601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:37.972934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d2659-a3b0-49f2-9920-89722172ae84 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:37.994233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.012834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8e88e-f5bb-4b8e-a2c7-4ad76ddb45f7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.014693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.015527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.015868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.016197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.017731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:38.018156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.019613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864cf54-d6f4-4a95-9cc5-de5d780b9089 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.020956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.041869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 399/794] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:38.050652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec56675-00c0-456e-b377-3a3e9761fb32 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.051112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec56675-00c0-456e-b377-3a3e9761fb32 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.051404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec56675-00c0-456e-b377-3a3e9761fb32 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.053112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 400/795] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:38.053918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec56675-00c0-456e-b377-3a3e9761fb32 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.057431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.059385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.059739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.060085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.070741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.073115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a1493420-8a68-4fc4-b540-2587a6391b77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:38.093597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:38.094599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:38.095697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 396/796] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:38.100620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.100620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.100620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.121814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:38.123146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:38.189497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec56675-00c0-456e-b377-3a3e9761fb32 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.198958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 397/797] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:38.200023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.227621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.228553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.228863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.229137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.239540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.241008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3ff4e2a6-3b0b-4942-a4fe-bf854cd301c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:38.258690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:38.259612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:38.260487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:38.260487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:38.260487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:38.260487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:38.260487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 401/798] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:38.261232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3be7f4-20b9-4f32-a30f-5c1443c59b35 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.261513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3be7f4-20b9-4f32-a30f-5c1443c59b35 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.261748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3be7f4-20b9-4f32-a30f-5c1443c59b35 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.267023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 402/799] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:38.267023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3be7f4-20b9-4f32-a30f-5c1443c59b35 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.294247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.307365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.314463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636085f-6ae2-4244-b2d9-4a6ac457c3cf None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.349857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.350623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.350963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.351300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.352141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:38.352444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.353913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.376174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 398/800] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:38.376174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.382878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3be7f4-20b9-4f32-a30f-5c1443c59b35 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.420460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.421320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.421597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.421876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.422543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.424197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-0ece1123-5f84-4df1-ae16-fa8bb92fcd0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:38.436545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:38.437529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:38.438379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 403/801] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:38.440266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.440266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.440266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.459317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:38.460590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:38.492590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48604e-260f-4a85-a19d-6efd1132cdd1 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.527116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.527793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.528074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.528348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.529103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:38.529419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.530785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.533982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 404/802] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:38.535147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.551994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 399/803] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:38.553242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.569845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.576600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.657046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fe59c-0950-41cd-a07a-194dc5c1e0bf None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.676096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efcb0b-2eda-4e2f-8d0d-59480ec3621d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.692932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.693749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.694378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.694647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.706577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.708570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.716958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.717663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.717943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.718217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.718931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:38.719236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.720868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.730285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 400/804] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:38.730285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:38.731114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:38.731955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:38.733213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:38.733213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:38.733213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:38.733213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:38.733213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 405/805] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:38.733733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.810048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.817103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:38.839533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb3327-343a-4261-953b-b6f0ce1fafe4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.847140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bbaf-1384-41ef-96a0-3b92283dc4e8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:38.874172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.874983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.875264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.875539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.882712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:38.883733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:38.884172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:38.885138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:38.885595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:38.886100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:38.896961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:38.898986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:38.899854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:38.900767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 401/806] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:38.901627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:38.907932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:38.908888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:38.909745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:38.914738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 406/807] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:38.915628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.008076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df35c39-ad50-4581-b6d6-15aad7d8a8d6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.014231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32c25-25c7-4bf0-8b4c-df776b5ce777 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.041219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.041885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.042163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.042439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.043148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:39.043460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.043594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.045280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.049120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.049807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.050082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.050341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.050576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 402/808] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:39.051146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:39.051367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.051524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.052917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.053877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.065356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 407/809] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:39.066104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.155008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ce30d-d5da-46fb-b3ca-6d88da779526 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.164662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384375a-12c0-4623-ba39-6607428de3af None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.194245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.194913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.195193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.195471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.196258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:39.196565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.198045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0d316-5af2-4e91-b92e-4472454ca8d5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.204905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.205580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.205856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.206120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.206868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:39.207170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.208518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.209805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 403/810] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:39.210961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac950043-ed2d-460d-9741-c3a56ec119b7 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.211246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac950043-ed2d-460d-9741-c3a56ec119b7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.211526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac950043-ed2d-460d-9741-c3a56ec119b7 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.212770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 404/811] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:39.213445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac950043-ed2d-460d-9741-c3a56ec119b7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.223720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 408/812] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:39.224539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.278961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.279542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.318331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac950043-ed2d-460d-9741-c3a56ec119b7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.325975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb944d5-7410-4272-a3ce-6e4cfd759691 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.352038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.352730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.353012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.353297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.354058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.355450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.359903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.360766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.361018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.361297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.369088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.370617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.376753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 405/813] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:20:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:39.381568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-fd796932-8080-4e63-a177-4c735debd6a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:39.385777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:39.386761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:39.387694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 409/814] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:39.388367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.388667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.388947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.408265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:39.409547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:39.495015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 410/815] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:39.495908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.502315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff845fe-0ccc-47ed-b2f9-8c279cf1b092 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.523184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.543443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.544762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.545219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.545663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/a4dec05937f84eedbbd5795582a16964/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.557072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.568244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=a4dec05937f84eedbbd5795582a16964, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.570275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.588366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 406/816] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:39 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/a4dec05937f84eedbbd5795582a16964/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:39.589614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.632879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9317979-a37b-423c-beb0-9367fd42de9d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.676869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.678141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.678633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.679218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.693014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.695513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:39.718118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:39.719088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:39.720275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:39.720275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:39.720275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:39.720275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:39.727877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ffa6-0a07-40d4-aee2-45d8473cf7e8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.728295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 411/817] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:39.729647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.771547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.772228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.772508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.772813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.773618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:39.773922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.775394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.787696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 407/818] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:39.788611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.820713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.839851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:39.851424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241387cd-2a22-4655-ba7c-6d3537d2bd3d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.894038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.894856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.895134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.895407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] PATH_INFO: `/v3/services/network` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.896061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.897491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.898495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24683c3f-e152-4e5d-ae66-0230ed90fd41 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:39.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:39.914916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:20:39.915935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:39.918810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 412/819] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:39.919993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:39.935862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:39.936727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:39.937147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:39.937514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:39.938400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:39.939596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:39.940761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:39.955298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 408/820] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:39.957388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:40.046160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40da1ef-3edc-4378-a224-f7037db61761 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.054231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:40.067839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcec7ad-68ef-4ce1-bc57-828eb854f9bb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.085360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.086687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.087110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.087534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/b96cbdec0f56412fb62617f03e77e713/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.089842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=b96cbdec0f56412fb62617f03e77e713, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.091465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.105245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.105928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.106265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.106543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.107310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] RBAC: Adding query filter params (name=network) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:40.107626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.109029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.109547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 413/821] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:39 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/b96cbdec0f56412fb62617f03e77e713/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:40.110789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:40.120929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 409/822] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:40.126648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:40.133756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:40.241446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8974-bf97-4c2f-92cf-cccc4b62bafc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.282795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.284011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.284489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.284991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.301339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.303443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:40.323236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:40.324213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:20:40.325196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:40.327847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 414/823] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:20:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:40.335256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:40.370220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00ca40-e982-45dc-87db-e14b6fb33fae None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.413345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.413973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.414255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.414530 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.415250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:40.415557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.416996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.422891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 410/824] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:40.430218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:40.448422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:40.451736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:40.482643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233da14-10bb-44b6-891f-2d2ad312e49e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.528846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.530312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.530732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.531161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.532170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:40.532791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.534448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.550136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15f00-81ae-4a18-b466-7bf51c8894d2 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.564614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 415/825] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:20:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:40.583163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:40.598869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.599552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.599829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.600179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.600966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.602362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.644506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 411/826] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:20:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:40.725573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb389f-9376-4379-9891-a74d6f4a0313 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:40.769496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:40.776462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:40.777196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:40.777480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:40.777752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:40.864910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:40.866412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:40.972093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 416/827] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 12:20:40.977709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:41.058611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:41.098642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7970098-2118-42fc-8c45-21209795fa36 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:41.134013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:41.134823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.135106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.135372 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.149045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:41.151629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:41.173200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:41.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:41.175455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:41.175455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:41.175455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:41.175455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:41.175455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 412/828] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:41.181995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:41.215937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:41.290454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013171d3-4619-44a4-a229-fb3caf150333 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:41.326603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:41.327276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.327564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.327834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.328558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:41.328896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:41.330314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:41.342956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 417/829] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:41.348743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:41.460785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1c0b6-40ab-471f-8b6c-1c42a5bb6f74 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:41.484731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:41.496237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:41.497936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.498232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.498509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.505673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:41.507131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-d0c37a94-2df3-4467-bcb4-1f22f1a590f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:41.521690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:41.523776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:41.525984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 413/830] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:41.528776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:41.627978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:41.643871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2318fc-0a8b-4023-95ae-23c39ddfb26c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:41.673279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff4e33-6ad0-4ba5-9829-b6b0ae898b5f None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.674479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff4e33-6ad0-4ba5-9829-b6b0ae898b5f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.674479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff4e33-6ad0-4ba5-9829-b6b0ae898b5f None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.675694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 414/831] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:41.682773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.682773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.682773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.683293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:41.683293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.683293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.683293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.683970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:41.684283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:41.685948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:41.699910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 418/832] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:41.705909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:41.706062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:41.707495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:41.792650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 415/833] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:41.807236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:41.834164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a8083-e402-401f-9a7f-0d558fda666a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:41.871060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:41.872078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.872556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.873106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.874128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0ca71f32f61e41288cac452e58b4a1dc) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:41.874648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:41.876329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d86b2-563d-498a-bef4-9d28585b5bcc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:41.888733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 419/834] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:41 2022] GET /identity/v3/role_assignments?user.id=0ca71f32f61e41288cac452e58b4a1dc&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:41.888733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab095ac7-8357-47d9-95f8-e931bf0afce6 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.888733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab095ac7-8357-47d9-95f8-e931bf0afce6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.888733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab095ac7-8357-47d9-95f8-e931bf0afce6 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.888733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 420/835] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:41.892438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:41.892761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:41.893037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:41.910648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:41.911775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:41.922822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:41.958179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac37aa7-a4c3-4205-9656-a8f43f241545 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:41.983080 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:41.993064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 421/836] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:42.004892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.005715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.006006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.006296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.006975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.008426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.011158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:42.025730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:42.026903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:42.027754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 416/837] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:42.034777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.139869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ac3d8-9ee2-4fb9-b6ed-96f6ca604fe4 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.179306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c832ef-67db-472a-b7af-80412c5587fb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.181198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.181602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.185518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.185518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.198779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:42.209852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:42.211012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:20:42.212093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:42.215377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 422/838] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:42.221434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.226163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:42.227175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.228103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.228598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.229141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.230384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:42.230909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.232501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.238342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 417/839] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:42.245459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.290964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:42.354690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaacf6-fb87-4159-9ab7-4ae3eca2f9a9 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.356865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302b7e-2742-4c3a-95cb-0859ff94b8bd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.390605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.391385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.391642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.391990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.392902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:42.393197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.394776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.399985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 418/840] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:42.401364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.402251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.402661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.403059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.403957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:42.404420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.405994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fc5de-6b65-4212-96b8-f4e70d09d25e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.421018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 423/841] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:42.422734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.490219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9344f-8d80-4dd8-81f0-b7f5dfc9fb02 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.490535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9344f-8d80-4dd8-81f0-b7f5dfc9fb02 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.490783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9344f-8d80-4dd8-81f0-b7f5dfc9fb02 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.492535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 424/842] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:42.498221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b10510-c398-4149-a268-0911f807827f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.498507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b10510-c398-4149-a268-0911f807827f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.498791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b10510-c398-4149-a268-0911f807827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.516995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d2b10510-c398-4149-a268-0911f807827f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:42.518187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d2b10510-c398-4149-a268-0911f807827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:42.537483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f36b9-82c6-4c56-8a38-95f81a46cf26 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.545110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:42.578861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.580000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.580286 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.580977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.587131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:42.595284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.596961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.597865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 425/843] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:42.614643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:42.615548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:42.616462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:42.616462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:42.616462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:42.616462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:42.616462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 419/844] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:42.621642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b10510-c398-4149-a268-0911f807827f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.621768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.731064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b10510-c398-4149-a268-0911f807827f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.731972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022eacb-1a56-4bdc-a6b1-2a765734dc16 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.767723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.768395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.768700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.768977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.769428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.769704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:42.770011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.770879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.771358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.771462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.771930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.783549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:42.788238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 426/845] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:42.789530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:42.792483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:42.793460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:20:42.794301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:42.797012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 420/846] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:42.806361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:42.824153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:42.851276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:42.915435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49151649-e84e-449b-919d-daf948db5f9c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:42.954222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:42.955223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:42.955654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:42.956082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:42.957024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:42.957496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:42.959059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:42.973061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 421/847] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:42.975512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.024647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56351bd-1448-4f2a-aed4-226b6061840e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.061251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.062062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.062336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.062604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.064778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.074961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.076668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.090402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ee91b-efad-4a67-838b-e89c534376a0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:43.097156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:43.098297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:43.099428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:43.102123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 427/848] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:43.107701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.131674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.132846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.133313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.133756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.145640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.147515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:43.171114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:43.172049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:43.173044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:43.173044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:43.173044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:43.173044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:43.180567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 422/849] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:43.181592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.233894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2b8d2-a0e7-4757-841a-34472c3c8bed None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.258167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.281542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.282757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.283189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.283861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.284877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:43.285340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.285953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd69a0b-0c24-4b12-b588-6bc22237dd29 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.287056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.308535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 428/850] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:43.314360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.320349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.326495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.328405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.329179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.330072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.331169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:43.331570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.333233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.352298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 423/851] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:43.354241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.439971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026f1fc-14e9-4d6e-b609-9c1c83b51e36 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.456517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df4c9-965f-4888-9fc6-46b8ac6339cc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.477931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.478881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.479379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.479790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.490260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.491057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.491332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.491596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.498871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.501140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:43.515714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:43.517570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:20:43.518709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:43.520960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 424/852] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:43.523069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.535063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.567249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.569745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.587865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.642380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79db9-8f11-4dfa-ba5d-1c35b182538e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.678621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.679306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.679579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.679847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.680723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:43.681030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.682463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.705991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 425/853] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:43.706903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.714793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 429/854] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:20:43.723359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.804893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.824597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06620cd-b5a0-464f-967b-cc6161f8f441 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.834303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96600515-5b5c-4f8a-b66a-1cacdbcb7506 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:43.837808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:43.864830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.865931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.866393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.866889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.875297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:43.876184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:43.876463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:43.876760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:43.889879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.891566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:43.909554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:43.910491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:43.911390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:43.911390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:43.911390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:43.911390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:43.913639 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 430/855] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:43.919255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:43.962885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:43.964879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.063471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37571-d577-41d4-b4ea-99ef3abacf2a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.091192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.105105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.105105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.105105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.105105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.105517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:44.105517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.105642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.106083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 426/856] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:20:44.107136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.119821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.121543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 431/857] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:44.128657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.249895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ee3cf-3eb5-4ff9-9d7c-4f8cdcef4f2e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.291007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.292676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.292829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.293020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.301425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.303151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-273ce9f8-ec7a-4180-b6a8-3f4d7a204aa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:44.317943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:44.319248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:44.320160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 432/858] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:44.321174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801e9a2-549d-41c3-80c6-e994cddc1b78 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.321174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801e9a2-549d-41c3-80c6-e994cddc1b78 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.321316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801e9a2-549d-41c3-80c6-e994cddc1b78 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.322652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 433/859] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:44.326101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801e9a2-549d-41c3-80c6-e994cddc1b78 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.358015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b8720-4ef3-44c5-9253-247ae4b7fbb4 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.374551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.393334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.394144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.394425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.394694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.406527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.407970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-94820133-bb67-421d-ab74-7a3e37c24be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:44.426832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:44.427766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:44.428680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:44.428680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:44.428680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:44.428680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:44.428680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 427/860] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:44.429585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05432731-4652-4adc-925a-a0870af23d34 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.429585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05432731-4652-4adc-925a-a0870af23d34 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.429833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05432731-4652-4adc-925a-a0870af23d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.437692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801e9a2-549d-41c3-80c6-e994cddc1b78 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.446927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-05432731-4652-4adc-925a-a0870af23d34 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:44.448131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-05432731-4652-4adc-925a-a0870af23d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:44.473711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.474381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.474661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.474934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.475713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:44.476024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.477535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.489744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 434/861] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:44.489744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.513607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.524886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 428/862] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:44.525724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05432731-4652-4adc-925a-a0870af23d34 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.591683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932233be-8449-41ce-aeda-55e061e53676 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.599167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.623806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05432731-4652-4adc-925a-a0870af23d34 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.627065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.627733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.628009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.628278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.629026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:44.629333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.630693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b1799-65aa-43fc-9f17-1e68618ed68a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.642323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 435/863] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:44.643521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfccd1-f1da-4e00-ae67-62ec25a89434 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.643803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfccd1-f1da-4e00-ae67-62ec25a89434 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.644059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfccd1-f1da-4e00-ae67-62ec25a89434 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.645316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 436/864] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:44.645999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfccd1-f1da-4e00-ae67-62ec25a89434 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.661295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.662046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.662327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.662599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.663435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=a4dec05937f84eedbbd5795582a16964) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:44.663743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.665203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.670981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 429/865] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/role_assignments?user.id=a4dec05937f84eedbbd5795582a16964&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:44.671743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.736505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.778461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfccd1-f1da-4e00-ae67-62ec25a89434 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.786682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b498b-d052-4af4-9d38-fe2d93b59aec None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.816722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.817671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.817959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.818229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.819248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.820677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.823820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:44.824640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.825605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.826143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-a219ce80-6fd9-4c15-b454-db40b29fe306 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:44.833426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:44.834379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:44.835516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 437/866] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:44.836653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:44.836653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:44.836653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:44.839702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:44.841452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:44.855188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:44.856318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:44.857561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:44.858453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:44.859344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 430/867] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:44.859825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.891863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:44.952651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 438/868] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:44.952651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:44.970325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8f1e6-e4a6-4dbc-baf9-5c4cebe6114d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:44.986919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:45.006028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:45.006702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:45.006985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:45.007258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:45.007969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:45.008276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:45.009688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:45.014609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 431/869] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:45.015287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:45.063614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35350247-befc-4e30-a997-5d78df44c4e6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:45.100084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:45.100783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:45.101063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:45.101343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:45.102120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:45.102425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:45.103865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:45.116090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 439/870] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:45.116875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:45.123218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b255cd9-514f-43c0-b650-e8de9cdbf680 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:45.139640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:45.158213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:45.158886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:45.159161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:45.159431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:45.160152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:45.161576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:45.179204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 432/871] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:20:45 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:45.179806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:45.229166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:45.261228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd5f08-3691-4377-bb91-753a021507a0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:45.307556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27702e18-fa19-452a-9798-720b3406df16 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:45.312571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:45.313847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:45.314333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:45.314808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:45.315734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:45.316981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:45.319515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f4c05-c84a-442e-a428-ec42e122eead None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:45.327409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 440/872] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:45.350843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:45.351536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:45.351817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:45.352089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:45.352993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=b96cbdec0f56412fb62617f03e77e713) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:45.353306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:45.354808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54221132-6e5c-4463-b74a-10484122a5bf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:45.362689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 433/873] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:45 2022] GET /identity/v3/role_assignments?user.id=b96cbdec0f56412fb62617f03e77e713&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:46.787482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36382b-e8f6-487c-bf66-a25d9372e955 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:46.788050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36382b-e8f6-487c-bf66-a25d9372e955 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:46.788050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36382b-e8f6-487c-bf66-a25d9372e955 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:46.789578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 441/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:46.794975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:46.795786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:46.796253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:46.814788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:46.816297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:46.891642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 434/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:46.916292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36382b-e8f6-487c-bf66-a25d9372e955 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.011885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.030550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36382b-e8f6-487c-bf66-a25d9372e955 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.066569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809add00-5fca-4a34-aca6-50ddf1911983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.067310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809add00-5fca-4a34-aca6-50ddf1911983 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.067590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809add00-5fca-4a34-aca6-50ddf1911983 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.067873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809add00-5fca-4a34-aca6-50ddf1911983 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.080164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809add00-5fca-4a34-aca6-50ddf1911983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-809add00-5fca-4a34-aca6-50ddf1911983 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:47.088255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:47.089224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:47.090043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 442/876] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jun 30 12:20:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:47.097929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.129686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.216277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62775f0-a485-4c11-abd8-460292f369cd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.251070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.251710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.251958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.252232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.253013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:47.253324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.254722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.256032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f45d43-f6c7-434c-9141-8ebed063c388 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.256321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f45d43-f6c7-434c-9141-8ebed063c388 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.256599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f45d43-f6c7-434c-9141-8ebed063c388 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.258271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 443/877] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:47.263092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.263370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.263636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 435/878] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:47.263746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.271042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.285652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:47.286999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:47.382620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.384804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 444/879] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:47.401346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595032aa-89c1-4b9c-98bf-2b4e9a4b40de None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.410666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.443217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.444444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.445004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.445114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.457086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.458545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9bb4805a-1cfc-4281-a202-0907c6b32da9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:47.476689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:47.477821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:47.478739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:47.478739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:47.478739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:47.478739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:47.478739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 436/880] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:47.479182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002150-9964-4e3a-a270-32c9822de420 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.479456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002150-9964-4e3a-a270-32c9822de420 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.479729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002150-9964-4e3a-a270-32c9822de420 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.481243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 437/881] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:47.484584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36002150-9964-4e3a-a270-32c9822de420 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.498379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.522721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518e718-7655-4653-b3b0-bc22dc80502a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.557199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.558008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.558289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.558561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.559209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.560592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-167f0942-122a-4204-ad40-5611d8c95009 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:47.572462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:47.573537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:47.574372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 445/882] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:47.576812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.577099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.577375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.590797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36002150-9964-4e3a-a270-32c9822de420 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.593638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:47.594924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:47.626429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.627236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.627518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.627793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.628571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:47.628930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.630321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292927ca-b2f2-4ab8-8ec8-dba791604b79 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.641891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 438/883] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:47.644196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab99c8-cf7d-48bc-b1be-1ee2ba677aae None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.644620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab99c8-cf7d-48bc-b1be-1ee2ba677aae None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.644929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab99c8-cf7d-48bc-b1be-1ee2ba677aae None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.647480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 439/884] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:47.647480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab99c8-cf7d-48bc-b1be-1ee2ba677aae None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.651635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.663735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 446/885] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:47.664575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.730130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.749075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab99c8-cf7d-48bc-b1be-1ee2ba677aae None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.765127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e93e2-7272-4e71-a165-f1640a2a3894 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.783154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.783821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.784104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.784394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.785143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:47.785446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.786827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed33e6-4e95-40a9-8816-13cd53b334f7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.791470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 440/886] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:47.792798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.793009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.793273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.801503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.802324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.802601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.802871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.809436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:47.810019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.810626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:47.811457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:47.826597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:47.827636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 447/887] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:47.828619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.876151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.881824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 441/888] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:47.882666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:47.938718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13afba6-f4a3-4c85-bdf0-db6e4352edd8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.965624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:47.973897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:47.974570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:47.974849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:47.975115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:47.975836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:47.977315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:47.983854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28da27-35ae-4ff0-a240-8c49d917e274 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:47.991974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 448/889] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:20:47 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:47.992818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.038281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.039526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.040012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.040641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.041492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:48.041897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.043369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.050494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 442/890] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:48.051388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.113287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497510e-e817-45b6-9734-81d453b65c80 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.119069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:48.150866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.151847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.151847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.152131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.152945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:48.153862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.154681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.176745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7d8cc-a03f-436f-af46-6b8305b45c3a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.179295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 449/891] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jun 30 12:20:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:48.180555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.217722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.218704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.219009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.219734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.220692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.222320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.235683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:48.239009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:48.240004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:48.240887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 443/892] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:48.241687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.305725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cb0fe-04cf-44ca-8404-ea6b073566d1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.344962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.345623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.345901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.346163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.346941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] RBAC: Adding query filter params (interface=public, service_id=6da46fc0966c42bcb5458da5ef3aec01, region_id=RegionOne) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:48.347250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.348025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4f786-6c08-4e2f-86ac-adc419a73327 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.348622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.358652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 450/893] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/endpoints?service_id=6da46fc0966c42bcb5458da5ef3aec01&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:48.358652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.375559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:48.388563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.389255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.389533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.389808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.461718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcb090-6b79-4fdf-ba4a-cbcd76095bef None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.473718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:48.479521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.481329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.502689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.504210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.504910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.505437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.506566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:48.507424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.509582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.516558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 451/894] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:48.518490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.590846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 444/895] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:20:48.591741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.616326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:48.631130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b498ca-cce5-4128-a805-02bf013df2c1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.667192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.667957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.668302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.668823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.669424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.670947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.677665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 452/896] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:20:48.677836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.714607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c508dc4-5828-4322-8a5e-884a306ebc4a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.723532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:48.767205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.768488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.769338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.769826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.771902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:48.772562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.774324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.782442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 445/897] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:48.810534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df87ca2-312c-4cd8-a130-1532ef418c26 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:48.851717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:48.852534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:48.852862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:48.853120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:48.865725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:48.867225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:48.887182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:48.888238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:48.889156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:48.889156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:48.889156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:48.889156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:48.891203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 453/898] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:48.896725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:48.991868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:49.008070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554eebd3-7f56-4329-903e-b42edf3315b9 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:49.048641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:49.049240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:49.049517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:49.049795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:49.050571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:49.050878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:49.052296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:49.064604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 446/899] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:49.070447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:49.174834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:49.183313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf3ed3-619f-47d4-b8b7-6fd455cadaa6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:49.219827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:49.220558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:49.220866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:49.221185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:49.228323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:49.229807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:49.243715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:49.244656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:49.245894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 454/900] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:49.252486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:49.355966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1795f91-d4e8-4c43-9e74-a0d6f1d4faed None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:49.390452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:49.391120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:49.391496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:49.391751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:49.392540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:49.392879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:49.394313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80225d-955a-46be-9b8b-1944b1efeeca None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:49.406934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 447/901] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:49.413753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:49.517206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671649f-8592-480f-9787-d5ba7d55433a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:49.556560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:49.557236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:49.557514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:49.557791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:49.558613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] RBAC: Adding query filter params (role.id=5907c6a5fd9d42d5985eacd0cf4a6706, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=a4dec05937f84eedbbd5795582a16964) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:49.558921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:49.560351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522ec92-2eec-4fa4-9a2f-d34b301e9c79 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:49.566707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 455/902] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:49 2022] GET /identity/v3/role_assignments?user.id=a4dec05937f84eedbbd5795582a16964&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=5907c6a5fd9d42d5985eacd0cf4a6706 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:50.116928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e7c1a-49c7-44bc-8e3f-2932c1ce9018 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.117470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e7c1a-49c7-44bc-8e3f-2932c1ce9018 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.117752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e7c1a-49c7-44bc-8e3f-2932c1ce9018 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.119484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 448/903] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:50.124800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.125200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.125524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.143712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:50.144936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:50.154459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:50.215316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 456/904] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:50.237947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e7c1a-49c7-44bc-8e3f-2932c1ce9018 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:50.247225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:50.350159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e7c1a-49c7-44bc-8e3f-2932c1ce9018 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:50.384347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:50.385120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.385399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.385670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.386328 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:50.387710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:50.400024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:50.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:50.402236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 449/905] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Jun 30 12:20:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:50.407608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:50.509080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e8490-f7da-475c-bb79-71dedf21d19b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:50.545627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:50.546305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.546583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.546859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.547559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:50.548037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:50.549317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:50.554630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 457/906] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:50.560393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:50.631735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:50.663718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8365feb-aa68-4ab9-b77c-98c22a8d049b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:50.701417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:50.702090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.702368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.702634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.703344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:50.703646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:50.705235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af469090-ac43-408c-81ae-1fabdae3166f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:50.710658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 450/907] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:50.716525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:50.727907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:50.760682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b806-e0cd-403e-8785-8f7cf8c0c82e None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.761031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b806-e0cd-403e-8785-8f7cf8c0c82e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.761286 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b806-e0cd-403e-8785-8f7cf8c0c82e None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.763272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 451/908] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:20:50.768945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.769273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.769496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.789820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:50.791313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:50.830855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db078-7038-4de8-a3c9-596408c91081 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:50.874460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:50.875162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:50.875408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:50.875681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:50.876523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] RBAC: Adding query filter params (interface=public, service_id=588e33ac0cc94010830fc1833d3980fe, region_id=RegionOne) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:50.876864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:50.878304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:50.880720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 452/909] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:50.897307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 458/910] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:20:50 2022] GET /identity/v3/endpoints?service_id=588e33ac0cc94010830fc1833d3980fe&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:50.902627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:50.902810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:50.967522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:50.975519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:51.007724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc7e91-cfa1-407f-a3a1-5db010f21ad7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.008013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43efea8-8904-4d0c-825f-1cfeca936cde None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.043539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88da26-47e6-4bf0-9347-133f4c184d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.044207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88da26-47e6-4bf0-9347-133f4c184d54 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.044347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.044485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88da26-47e6-4bf0-9347-133f4c184d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.044794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88da26-47e6-4bf0-9347-133f4c184d54 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.045103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.045379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.045490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88da26-47e6-4bf0-9347-133f4c184d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:51.045648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.046303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:51.046907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88da26-47e6-4bf0-9347-133f4c184d54 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:51.047829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:51.053111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 459/911] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:20:50 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:20:51.054556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956d5b5-56a5-40d6-bb30-e70f9f24fd9d None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.055017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956d5b5-56a5-40d6-bb30-e70f9f24fd9d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.055462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956d5b5-56a5-40d6-bb30-e70f9f24fd9d None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.057225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 460/912] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:51.062697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.062972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.063252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:51.064995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:51.066050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:20:51.066949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:51.070125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 453/913] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:51.076288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:51.082522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:51.084064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:51.189643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 461/914] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:51.206132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:51.218947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873ce86-3798-4316-a914-241a5338bf0b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.250646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:51.260351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.261085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.261362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.261631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.262344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:51.262656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:51.264037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:51.271027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 454/915] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:51.276885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:51.297107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:51.347016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93617-1dcd-4747-864f-f055a5d0089e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.385641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250941-69fa-4213-9611-c0d38f88965c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.395731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.396875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.397156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.397429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.398576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:51.400241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:51.419214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 462/916] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:20:51 2022] POST /identity/v3/services => generated 241 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:51.439234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.440418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.440952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.441427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.442388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:51.442971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:51.445079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f0c2-2181-4e0a-ae5b-c355b23c7cb6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:51.451991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 455/917] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jun 30 12:20:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:51.457898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:51.575128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90c08c-88f5-4235-9e0f-0752a999bb06 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.604354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:51.616792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd646437-5188-4d79-8978-f3ef663dacdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.617521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd646437-5188-4d79-8978-f3ef663dacdc None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.617806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd646437-5188-4d79-8978-f3ef663dacdc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.618080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd646437-5188-4d79-8978-f3ef663dacdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.618841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd646437-5188-4d79-8978-f3ef663dacdc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:51.620229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd646437-5188-4d79-8978-f3ef663dacdc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:51.643482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 463/918] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:20:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:51.665458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369400b6-b667-4650-a8ad-da08dd09ae41 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.666124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369400b6-b667-4650-a8ad-da08dd09ae41 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.666618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369400b6-b667-4650-a8ad-da08dd09ae41 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.668698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 456/919] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:51.675179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.675550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.675810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.694799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:51.696028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:51.780782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 464/920] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:51.805152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369400b6-b667-4650-a8ad-da08dd09ae41 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:51.863932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:51.914328 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369400b6-b667-4650-a8ad-da08dd09ae41 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:51.950699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:51.951919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:51.952418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:51.952929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:51.964935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:51.973668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:51.974636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:20:51.975473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:51.978215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 457/921] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jun 30 12:20:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:51.984503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:52.086777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a0bb9-9eb5-4be2-9075-9e2c3534ff2d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:52.097040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:52.126077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:52.127134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:52.127654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:52.128138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:52.129147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:52.129675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:52.131272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:52.140791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 465/922] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:20:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:52.146861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:52.253082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b8491-6d39-483f-a8e7-d08b1cea67b3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:52.267137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:52.290637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:52.291450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:52.291730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:52.292005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:52.302459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:52.303909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:52.321781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:52.322794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:52.323685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:52.323685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:52.323685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:52.323685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:52.327244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 458/923] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:52.329386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:52.433736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717dec-a040-4184-b325-d469a7b97de7 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:52.467882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:52.468934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:52.469434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:52.469959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:52.470884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:52.471413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:52.473020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:52.485739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 466/924] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:52.491345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:52.588434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec7f06-05d2-4b41-a9d6-b5c3667cd4cf None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:52.621871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:52.623038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:52.623538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:52.624045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:52.632886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:52.634744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:52.651499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:52.652567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:52.653566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:52.659977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 459/925] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:20:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:52.662903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:52.694762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:52.767758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caacd6-9ccf-44d0-aec1-92a0c3eecc8f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:52.803420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:52.804474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:52.805090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:52.805569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:52.806600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:52.807114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:52.808789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:52.827673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 467/926] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jun 30 12:20:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:52.834046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:52.861003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:52.940102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa129-c975-4044-9b7a-0766e5cdf18a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:52.975298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:52.976540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:52.977057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:52.977540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:52.988840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:52.990539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:53.008106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:53.009050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:53.009938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:53.009938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:53.009938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:53.009938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:53.015530 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 460/927] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:53.018403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.109547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:53.124250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a55b3-cb36-4d67-8646-5d56d87c1058 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.160056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.161175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.161671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.162176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.163096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:53.163597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.165213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2c67c-7f21-4329-98c3-99bdcb9e12c5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.180756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 468/928] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:53.190913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.222815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8739d-60b9-4c78-ab55-4a22b17971e4 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.223130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8739d-60b9-4c78-ab55-4a22b17971e4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.223369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8739d-60b9-4c78-ab55-4a22b17971e4 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.225169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 469/929] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:20:53.230089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.230316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.230596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.248285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:53.249518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:53.287324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:53.312994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14125fc-328c-49e1-bfb4-8b598e2e92f8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.340660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 470/930] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:20:53.359930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.360875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.361160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.361434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.362450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.368582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.370116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:53.386225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:53.387175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:53.393760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 461/931] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:53.396938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.464668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:53.489955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80468897-7785-410b-b9d0-b7047d5ddc8e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.512426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c13b2-5019-4258-84d6-61e2fa000e7d None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.527059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.528261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.528774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.529248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.530102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.531672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:53.545157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:53.546196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:20:53.547022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:53.550709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.554763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.554763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.554763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.554763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:53.554763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.554763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.555315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 471/932] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:53.556717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.557404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:53.574606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 462/933] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:53.577088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.681371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbdecf-a98f-40d3-b7bb-5f73c6db02fd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.704728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f164ad-2c87-433a-8cc4-5232c7972486 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.717298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.717970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.718261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.718518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.719224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:53.719587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.720992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896d1f-6fc3-453f-9525-c234b1b4294d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.730477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 472/934] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:53.730477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068b6af-8b23-4898-973e-a3fa903a40a9 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.730477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068b6af-8b23-4898-973e-a3fa903a40a9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.730477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068b6af-8b23-4898-973e-a3fa903a40a9 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.733350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 473/935] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:53.733979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068b6af-8b23-4898-973e-a3fa903a40a9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:53.743775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:53.746560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdd32-a273-4a0e-9a49-23e0ce28f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.747699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdd32-a273-4a0e-9a49-23e0ce28f327 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.748071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdd32-a273-4a0e-9a49-23e0ce28f327 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.748556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdd32-a273-4a0e-9a49-23e0ce28f327 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/a4dec05937f84eedbbd5795582a16964/roles/5907c6a5fd9d42d5985eacd0cf4a6706` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.752928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcdd32-a273-4a0e-9a49-23e0ce28f327 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=a4dec05937f84eedbbd5795582a16964, role_id=5907c6a5fd9d42d5985eacd0cf4a6706)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.754335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcdd32-a273-4a0e-9a49-23e0ce28f327 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.771052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 463/936] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:20:53 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/a4dec05937f84eedbbd5795582a16964/roles/5907c6a5fd9d42d5985eacd0cf4a6706 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:20:53.772728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae847f-7ab5-467d-9d1c-63f32ddfb2a9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.772962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae847f-7ab5-467d-9d1c-63f32ddfb2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.773235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae847f-7ab5-467d-9d1c-63f32ddfb2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.791989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-99ae847f-7ab5-467d-9d1c-63f32ddfb2a9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:53.793248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-99ae847f-7ab5-467d-9d1c-63f32ddfb2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:53.864132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:53.867108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068b6af-8b23-4898-973e-a3fa903a40a9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:53.893282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 464/937] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:53.894670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cef68-d0f3-4af4-a8c4-08fee83207a8 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.894960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cef68-d0f3-4af4-a8c4-08fee83207a8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.895233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cef68-d0f3-4af4-a8c4-08fee83207a8 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.896724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 465/938] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:53.903147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.903463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.903701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.918804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:53.920131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:53.920747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:53.921251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:53.922158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:53.922873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:53.923640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:53.924743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:53.924908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:53.931892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 474/939] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:53.933103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.006343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 466/940] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:54.007460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.026574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:54.062068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cbb73-e866-40ee-81fc-e2f473c0c387 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.099240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.100354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.100869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.101301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.102146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.103574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:54.117064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:54.118115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:54.118965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.119720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6faf1-c074-43c4-8cf1-102bd05ae140 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.119798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 475/941] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:54.119798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.140577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:54.159966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.160654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.160950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.161221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.161961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.163348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.200672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 467/942] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:20:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:54.200672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.253857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d5dc-125d-4c1c-a094-5a5562a7a278 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.299859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:54.302815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.303990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.304473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.305011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.306103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.308007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.309920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING keystone.server.flask.common [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 12:20:54.310564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2b1bc-554c-432e-8618-8337852f7daf None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.340401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.sql.core [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 12:20:54.340401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 12:20:54.340401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [parameters: {'id': 'c24b71847ba8439a93c66c5641c7b801', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 12:20:54.340401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80096) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 12:20:54.350307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.350992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.351274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.351547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.352343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:54.352676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.354179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.360544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 468/943] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:20:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:54.366003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c24b71847ba8439a93c66c5641c7b801.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c24b71847ba8439a93c66c5641c7b801. Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:20:54.369395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result.read() Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 12:20:54.370469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application yield resource Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise self.value Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application session.commit() Jun 30 12:20:54.371468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.session.flush() Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application compat.raise_( Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 12:20:54.372357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 12:20:54.373286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result.read() Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:20:54.374098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [parameters: {'id': 'c24b71847ba8439a93c66c5641c7b801', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.374965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:54.375854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c24b71847ba8439a93c66c5641c7b801. Jun 30 12:20:54.377681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.380708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 476/944] 10.222.0.57 () {60 vars in 1140 bytes} [Thu Jun 30 12:20:54 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:54.386824 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.416028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:54.502499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c441c3-6f6b-4f18-add4-61c0e250cc43 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.514923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b5bd0-f53a-4d6e-a0f9-0e2227dae08f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.540018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.541020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.541448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.541936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.542902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:54.543349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.545014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.550781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.551511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.551651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 469/945] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:20:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:54.551792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.552065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.558731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.564435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.566420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:54.588718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:54.589648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:54.590570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:54.593272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 477/946] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:54.599092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.647470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:54.680654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:54.684522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e133a-18ed-479f-8b09-0c3defa36782 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.704241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c9d2c-3cd1-475f-af46-6fa7252e0034 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.724344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.726612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.727037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.727464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.728527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] RBAC: Adding query filter params (interface=public, service_id=824c091da05540708b71136bd8e549a3, region_id=RegionOne) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:54.729033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.730704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.738612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 470/947] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:20:54 2022] GET /identity/v3/endpoints?service_id=824c091da05540708b71136bd8e549a3&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:54.742744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.743631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.743913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.744182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.744977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:54.745281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.746023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:54.746820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb687a-f5d3-4e5c-b10a-83de71aab26c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.776496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 478/948] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:54.864972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc8b8f-635f-4c9e-b229-8dfa75a65368 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:54.902326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a6dad-061b-4ab1-b086-adc8a7c828c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:54.903371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6dad-061b-4ab1-b086-adc8a7c828c7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:54.903853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6dad-061b-4ab1-b086-adc8a7c828c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:54.904339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6dad-061b-4ab1-b086-adc8a7c828c7 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:54.905265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a6dad-061b-4ab1-b086-adc8a7c828c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:54.907082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a6dad-061b-4ab1-b086-adc8a7c828c7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:54.915357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 471/949] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:20:54 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:20:55.888808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d1a20-24b6-41ab-94fa-e95588c71652 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:55.889346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d1a20-24b6-41ab-94fa-e95588c71652 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:55.889346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d1a20-24b6-41ab-94fa-e95588c71652 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:55.891021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 479/950] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:55.898029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:55.898487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:55.898794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:55.916161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:55.917404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:55.941669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:55.991509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 472/951] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:56.028415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d1a20-24b6-41ab-94fa-e95588c71652 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.197685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d1a20-24b6-41ab-94fa-e95588c71652 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:56.207243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:56.235707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cd628-8e4a-4245-9fa1-6b8979efcd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:56.236835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cd628-8e4a-4245-9fa1-6b8979efcd78 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.237178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cd628-8e4a-4245-9fa1-6b8979efcd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.237510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cd628-8e4a-4245-9fa1-6b8979efcd78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.247976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cd628-8e4a-4245-9fa1-6b8979efcd78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c88cd628-8e4a-4245-9fa1-6b8979efcd78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:20:56.256192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:56.257262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:56.258102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 480/952] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:56.263750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.372188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70111f4a-480c-41ae-8146-896fcc78a033 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:56.405584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964e53-bb8e-43bc-9f1c-ec9874bb39a8 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.405922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964e53-bb8e-43bc-9f1c-ec9874bb39a8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.406119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964e53-bb8e-43bc-9f1c-ec9874bb39a8 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.407903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 481/953] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:56.413004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.413445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.413794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.420715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:56.422252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.422637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.427937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.428801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:56.429186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:56.430660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:56.443017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:56.447433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:56.450007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 473/954] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:56.453625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.497201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:56.599365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 482/955] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:56.604648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640535e-7e93-4dd7-b246-0e32441e7f54 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:56.634326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.639953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:56.640905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.641254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.641761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.653280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:56.655054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:56.676656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:56.677783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:56.678851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:56.678851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:56.678851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:56.678851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:56.678851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 474/956] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:56.684645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.783331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:56.789803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e08a0-dcc1-4cab-83cb-d4e73a2784de None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:56.805931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:56.830623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:56.831722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.832263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.832707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.833572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:56.835209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:56.839002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60601758-0a20-4017-a0f9-1142d9db1c98 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:56.847972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:56.849280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:20:56.850298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:56.852888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 483/957] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:56.858716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.875234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:56.875946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:56.876226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:56.876499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:56.877333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:56.877653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:56.879512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:56.893345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 475/958] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:56.900729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:56.975758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc24e46-6fca-4497-9df4-618d4d14125c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.001748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f437e-c702-414b-bb83-442a093c8fc0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.011490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.012163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.012448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.012755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.013465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:57.013778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.015158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88a55-0b9c-4604-829a-671f6034d469 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.020308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 484/959] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:20:57.021756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e061-dd63-4a9d-ab95-0d16d41b3817 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.022046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e061-dd63-4a9d-ab95-0d16d41b3817 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.022341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e061-dd63-4a9d-ab95-0d16d41b3817 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.023566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 485/960] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:57.025823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e061-dd63-4a9d-ab95-0d16d41b3817 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.037472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.044113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.044971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.045252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.045524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.053330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.054779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.058188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-de069bf3-0519-454a-aee7-5773ac90a4fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:57.071802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:57.072916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:57.073876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 476/961] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:20:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:57.074537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c756cdb-94f6-463a-85af-e19c1982d27f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.074617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c756cdb-94f6-463a-85af-e19c1982d27f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.074932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c756cdb-94f6-463a-85af-e19c1982d27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.091305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-1c756cdb-94f6-463a-85af-e19c1982d27f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:57.092488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1c756cdb-94f6-463a-85af-e19c1982d27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:57.135524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e061-dd63-4a9d-ab95-0d16d41b3817 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.157964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 477/962] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:57.159265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806f63c-ffee-4060-9869-753493cc9a6d None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.159548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806f63c-ffee-4060-9869-753493cc9a6d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.159820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806f63c-ffee-4060-9869-753493cc9a6d None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.161119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 478/963] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:57.161809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806f63c-ffee-4060-9869-753493cc9a6d None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.179480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.180158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.180436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.180744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.181453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:57.181755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.183120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48ce6b-8016-4695-b548-3e010262dc16 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.191957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 486/964] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:57.191957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.191957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.191957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.208902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:57.210128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:57.292952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 487/965] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:57.292952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.305331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806f63c-ffee-4060-9869-753493cc9a6d None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.322238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.333112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.346237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.346945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.347226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.347500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.348347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:57.348686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.350142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.370311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 479/966] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:57.371148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.418383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293e804-e94b-49d7-87bb-337e4e00729f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.456249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.456965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.457247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.457526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.458223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.459660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.473965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.sql.core [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 12:20:57.473965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 12:20:57.473965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [parameters: {'user_id': '721c187e40054262844730524490f5aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 12:20:57.473965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80096) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result.read() Jun 30 12:20:57.482083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:20:57.482999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application yield resource Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise self.value Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application session.commit() Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 12:20:57.483959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.session.flush() Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application compat.raise_( Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 12:20:57.485675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:20:57.486654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application result.read() Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:20:57.488260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [parameters: {'user_id': '721c187e40054262844730524490f5aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:57.490323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 12:20:57.492382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 488/967] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jun 30 12:20:57 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:57.495638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.496801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9171-9b5b-4ce4-9cfb-b8de2cbfc9b2 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.535371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.536182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.536459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.536774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.537432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.538802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:57.550990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:57.552061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:57.552922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 480/968] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:57.553723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.593401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.602178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.618026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d938070-d3f1-40e8-97a6-fea401acfdef None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.655850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.656849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.657434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.657812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.662982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf917e-82d9-41bf-b1f6-2ddc71e770c7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.698568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.699379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.699658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.699925 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.707903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.709390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:20:57.723796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:57.725016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 481/969] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:57.725941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.735016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.736798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.824096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.831165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231f21-38f6-457d-9809-e9b43c94e8ac None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.834537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:57.851342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 489/970] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:20:57.851342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.876087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:57.877065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:57.877388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:57.877695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:57.878384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:57.878687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:57.880078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:57.885067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 482/971] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:57.885851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:57.977056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25b73-d576-4a28-8426-1241e388eb68 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:57.990157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083dbbc-51c7-4863-9c5f-83e533e420f0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.018142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.018815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.019097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.019417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.020148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:58.020451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.021937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.026859 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.027666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.027949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.028221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.039221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.040695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:20:58.058207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:58.059092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:58.059983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:20:58.059983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:20:58.059983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:20:58.059983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:20:58.059983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 483/972] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:20:58.060787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:58.061834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 490/973] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jun 30 12:20:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:58.073396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:58.086341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:58.137444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:58.188034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4d34f-0d38-4b2d-ba20-c7774ac8a91a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.212511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537451d-7ccd-48e1-8d79-f1bee917c67c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.237058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.238057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.238336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.238606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.239331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:58.239630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.241826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.247717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 484/974] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:20:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:58.251082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.252134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.252414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.252787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.253599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:58.253911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.254231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:58.255287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.268079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 491/975] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:58.273224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:58.372361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:58.396403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:58.399745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724bf56-4e72-49f8-ac5e-17724d712439 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.400285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c281936-3727-4f50-a55a-7c2e2e493553 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.434381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.435186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.435467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.435744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.438100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.438874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.439154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.439424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.440311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.441749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.442835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.444664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:20:58.459537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:20:58.460464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:20:58.466327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 492/976] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:20:58.466847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 485/977] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:20:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:20:58.470062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:58.582713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf6095-3913-457b-bfa3-1a90b9ba734b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.632998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.633593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.633875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.634151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.634935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:58.635251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.636707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322c82f-c0f7-488c-a926-5e9e4d2a45d1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.649577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 493/978] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:58.655583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:58.760399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d271dd-3433-4cde-98de-c6de7777ccbc None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:58.796085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:58.796768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:58.797048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:58.797317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:58.798192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] RBAC: Adding query filter params (role.id=5907c6a5fd9d42d5985eacd0cf4a6706, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=a4dec05937f84eedbbd5795582a16964) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:20:58.798512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:58.799949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7bfab-6cad-47ba-b50f-10e524b8863c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:58.806253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 486/979] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:20:58 2022] GET /identity/v3/role_assignments?user.id=a4dec05937f84eedbbd5795582a16964&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=5907c6a5fd9d42d5985eacd0cf4a6706 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:20:59.230806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda173-951c-47c0-bed0-f1a022b03672 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:59.231347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda173-951c-47c0-bed0-f1a022b03672 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:59.231347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda173-951c-47c0-bed0-f1a022b03672 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:59.236703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 494/980] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:20:59.238653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0ca5-16ea-431d-9472-ad9ce3400062 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:59.239072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0ca5-16ea-431d-9472-ad9ce3400062 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:59.239246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0ca5-16ea-431d-9472-ad9ce3400062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:59.245651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:59.257448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-825d0ca5-16ea-431d-9472-ad9ce3400062 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:20:59.258645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-825d0ca5-16ea-431d-9472-ad9ce3400062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:20:59.327673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 487/981] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:20:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:20:59.351056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda173-951c-47c0-bed0-f1a022b03672 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:20:59.367883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:20:59.463451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda173-951c-47c0-bed0-f1a022b03672 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:20:59.497682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3be38-6ea6-4ed0-89a5-d32566494819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:20:59.498356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be38-6ea6-4ed0-89a5-d32566494819 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:20:59.498637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be38-6ea6-4ed0-89a5-d32566494819 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:20:59.498910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be38-6ea6-4ed0-89a5-d32566494819 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:20:59.499642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3be38-6ea6-4ed0-89a5-d32566494819 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:20:59.501169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3be38-6ea6-4ed0-89a5-d32566494819 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:20:59.524668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 495/982] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:20:59 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:00.539302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ecf71-cf92-453a-a2bd-8c3c6a822762 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:00.539895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ecf71-cf92-453a-a2bd-8c3c6a822762 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:00.539895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ecf71-cf92-453a-a2bd-8c3c6a822762 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:00.544665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 488/983] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:00.546814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:00.547217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:00.547385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:00.566121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:00.567645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:00.646740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:00.654241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 496/984] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:00.677529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ecf71-cf92-453a-a2bd-8c3c6a822762 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:00.783276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ecf71-cf92-453a-a2bd-8c3c6a822762 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:00.819912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:00.820676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:00.820961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:00.821234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:00.821892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:00.823266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:00.825193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-ccb366a9-0e0f-426b-b15d-fe7d0c9f59f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:00.836300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:00.837397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:00.843278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 489/985] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:21:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:00.846715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:00.875468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b866aa3-7cb9-43be-9cda-2afa6f287547 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:00.875826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b866aa3-7cb9-43be-9cda-2afa6f287547 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:00.876109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b866aa3-7cb9-43be-9cda-2afa6f287547 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:00.877602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 490/986] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:21:00.882422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:00.882708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:00.882982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:00.899581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:00.900804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:00.957878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c98c70-a7e2-4c64-bcf5-477d086f563a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:00.972268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 491/987] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:00.992230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:00.992938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:00.993219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:00.993780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:00.994489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:00.994587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:00.994795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:00.996167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.001954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 497/988] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:21:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:01.007608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.070977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:01.099731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9fcfe-9d7d-421c-969c-5546fc57e007 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.116474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ccf6b-4c4c-408a-9f90-967764e3c6d8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.128714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:01.138425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a835a-eb87-4538-8160-21fac351737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.138982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a835a-eb87-4538-8160-21fac351737b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.139265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a835a-eb87-4538-8160-21fac351737b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.139545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a835a-eb87-4538-8160-21fac351737b None admin] PATH_INFO: `/v3/services/image` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.140269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a835a-eb87-4538-8160-21fac351737b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.142074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a835a-eb87-4538-8160-21fac351737b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.151876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.152553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.152862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.153131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.153832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:01.154137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-992a835a-eb87-4538-8160-21fac351737b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:01.154893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:01.155830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:01.157669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:01.158409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.158482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 492/989] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Jun 30 12:21:00 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:21:01.164848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 498/990] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:01.168172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.168291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a835a-eb87-4538-8160-21fac351737b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.273798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9622572-7c56-40ff-b356-4b88ee73023e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.276227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a835a-eb87-4538-8160-21fac351737b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.301988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:01.311761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.312516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.312955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.313295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.313599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.314249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] RBAC: Adding query filter params (interface=admin, service_id=824c091da05540708b71136bd8e549a3, region_id=RegionOne) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:01.314344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.314539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.314640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.314811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.315519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:01.315823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.316253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.317235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.327423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 499/991] 10.222.0.57 () {56 vars in 1249 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/endpoints?service_id=824c091da05540708b71136bd8e549a3&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:01.327564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 493/992] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:01.328257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.328358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.387780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:01.433302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4600e1-f969-4f69-b7fa-b08df85d168d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.434450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41eb7-e6a5-4133-ae60-ce312ccaf4b3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.472889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfbab2-dda0-48db-80ad-32794416cace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.473516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.473668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfbab2-dda0-48db-80ad-32794416cace None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.473844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfbab2-dda0-48db-80ad-32794416cace None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.474084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfbab2-dda0-48db-80ad-32794416cace None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.474766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfbab2-dda0-48db-80ad-32794416cace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.475056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.475506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.475962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.476133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfbab2-dda0-48db-80ad-32794416cace None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.477212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] RBAC: Adding query filter params (type=image) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:01.477990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.479854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de26d2-3423-4623-b155-70dcc55b21fc None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.482510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 500/993] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:21:01.492684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 494/994] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:01.592617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dd952-cd29-432a-b8b7-30868c44e179 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.593155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dd952-cd29-432a-b8b7-30868c44e179 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.593393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dd952-cd29-432a-b8b7-30868c44e179 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.595689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 501/995] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:01.602168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.602710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.602710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.622391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:01.623687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:01.703076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 495/996] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:01.726937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dd952-cd29-432a-b8b7-30868c44e179 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.789183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:01.837336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dd952-cd29-432a-b8b7-30868c44e179 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:01.873733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:01.874453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:01.874728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:01.875001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:01.875710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:01.877135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:01.888970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:01.889935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:21:01.890805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:01.892218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 502/997] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:01.896847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:01.950822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:02.004780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f060e-bbc3-48b5-9d24-1e667958bb97 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:02.039003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:02.039674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:02.039953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:02.040225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:02.040958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:02.041298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:02.042829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:02.053614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 496/998] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:21:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:02.054999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:02.161539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cf5f-bd00-48d7-a423-b8c76895a86a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:02.182970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:02.198276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:02.198958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:02.199241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:02.199514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:02.200225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:02.200535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:02.201947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:02.207338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 503/999] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:21:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:02.213024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:02.315258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa046bdd-46a2-45db-a0ae-eda210745c06 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:02.329652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:02.352227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:02.352920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:02.353202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:02.353471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:02.354264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] RBAC: Adding query filter params (interface=public, service_id=400aa76ad1794161b243f36e706785f7, region_id=RegionOne) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:02.354573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:02.355945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e812b-87ca-41c5-9a69-b67d682296ff None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:02.361466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 497/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:21:02 2022] GET /identity/v3/endpoints?service_id=400aa76ad1794161b243f36e706785f7&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:02.366973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:02.471446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f6ff-0766-4cc7-8eef-13a91cec473b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:02.505204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160acc72-ad32-4c05-9731-58c0f7131574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:02.505866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160acc72-ad32-4c05-9731-58c0f7131574 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:02.506144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160acc72-ad32-4c05-9731-58c0f7131574 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:02.506410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160acc72-ad32-4c05-9731-58c0f7131574 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:02.507078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160acc72-ad32-4c05-9731-58c0f7131574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:02.508453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160acc72-ad32-4c05-9731-58c0f7131574 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:02.515051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 504/1001] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:21:02 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:21:03.534133 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f72552-48ac-4688-9617-a84d0b781aa7 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:03.537023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f72552-48ac-4688-9617-a84d0b781aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:03.537023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f72552-48ac-4688-9617-a84d0b781aa7 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:03.537023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 498/1002] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:03.541680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:03.542205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:03.542205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:03.559750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:03.561007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:03.576335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969a035-60ce-4311-a060-9645ff0c3e35 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:03.576788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969a035-60ce-4311-a060-9645ff0c3e35 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:03.577064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969a035-60ce-4311-a060-9645ff0c3e35 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:03.578462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 499/1003] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:03.584235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:03.584518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:03.584824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:03.601628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:03.602815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:03.609313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:03.635264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 505/1004] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:03.658790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:03.677535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 500/1005] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:03.700658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:03.766368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26510b-feea-4724-a802-81bc85767ed2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:03.803033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d267-d0bb-45f3-ad37-56e9eff4112f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:03.803932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:03.804614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:03.804939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:03.805218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:03.805922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:03.807296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:03.813463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:03.824612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 506/1006] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:21:03 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:21:03.853951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:03.854789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:03.855065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:03.855332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:03.856004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:03.857443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:03.870672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:03.871864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:03.878441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 501/1007] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jun 30 12:21:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:21:03.881406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:03.926556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:03.998530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc732-9cd7-4edd-8e54-83af8cd7a646 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:04.033410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:04.034087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:04.034364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:04.034690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:04.035395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:04.035817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:04.037252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:04.043660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 507/1008] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:21:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:04.048930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:04.151606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f9982-74c3-4828-9f5c-6e25aa9e9bcd None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:04.193985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:04.194666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:04.194947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:04.195222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:04.195932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:04.196233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:04.197641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac793596-d2e2-4589-a426-71b7d7336854 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:04.203007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 502/1009] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jun 30 12:21:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:04.208947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:04.309417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39fee3-2f4c-4dd6-91a7-28a6f45df542 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:04.310983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:04.345282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790f674-0475-416a-a43d-47910c51f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:04.345946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790f674-0475-416a-a43d-47910c51f5d5 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:04.346226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790f674-0475-416a-a43d-47910c51f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:04.346497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790f674-0475-416a-a43d-47910c51f5d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:04.347245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790f674-0475-416a-a43d-47910c51f5d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:04.348622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790f674-0475-416a-a43d-47910c51f5d5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:04.368667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 508/1010] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:21:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:04.557522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e5b3-aa66-45c0-bc22-868490cdb1ff None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:04.558067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e5b3-aa66-45c0-bc22-868490cdb1ff None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:04.558067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e5b3-aa66-45c0-bc22-868490cdb1ff None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:04.559448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 503/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:04.564864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:04.565326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:04.565418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:04.582742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:04.583937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:04.658892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 509/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:04 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:04.681977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603e5b3-aa66-45c0-bc22-868490cdb1ff None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:04.715732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:04.794276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603e5b3-aa66-45c0-bc22-868490cdb1ff None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:04.830923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:04.831646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:04.831930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:04.832202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:04.832887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:04.834275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:04.846836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:04.847975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:04.848891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 504/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:21:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:04.856211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:04.951679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:04.966301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb1e54-2824-4402-89de-c258e16b286e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:05.002707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:05.003364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:05.003632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:05.003913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:05.004696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:05.005007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:05.006455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:05.016306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 510/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:21:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:05.017781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:05.117847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:05.124972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208e43c-a61f-43ae-9dad-dab60acbfc01 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:05.159947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:05.160616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:05.160942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:05.161331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:05.162064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:05.162373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:05.163757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff84fd-88e4-4422-9cbe-0460b78571a1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:05.180134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 505/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jun 30 12:21:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:05.181725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:05.283855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00564a9c-7afc-4139-b066-e7d82d3ba8cd None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:05.320105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdeb4b-0e7b-4ad8-a79f-9f2e234d7457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:05.320801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdeb4b-0e7b-4ad8-a79f-9f2e234d7457 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:05.321085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdeb4b-0e7b-4ad8-a79f-9f2e234d7457 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:05.321356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdeb4b-0e7b-4ad8-a79f-9f2e234d7457 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:05.322093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdeb4b-0e7b-4ad8-a79f-9f2e234d7457 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:05.323474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdeb4b-0e7b-4ad8-a79f-9f2e234d7457 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:05.348670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 511/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:21:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:05.899723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e07000-ebb0-416a-980e-b09ed1efdee0 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:05.900290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e07000-ebb0-416a-980e-b09ed1efdee0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:05.900290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e07000-ebb0-416a-980e-b09ed1efdee0 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:05.902477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 506/1017] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:05.908405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:05.908937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:05.909036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:05.926770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:05.928011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:05.930188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:06.015167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 512/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:06.039750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e07000-ebb0-416a-980e-b09ed1efdee0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.151422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e07000-ebb0-416a-980e-b09ed1efdee0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.192998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.194171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.194450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.194727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.195380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.196787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:06.208606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:06.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:06.215831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 507/1019] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Jun 30 12:21:06 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:06.216660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.320677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d0dc7-cd19-450a-a760-6f1446e3167a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.356670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.357361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.357650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.357914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.358616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:06.358920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.360506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec290a-1183-41b3-b39b-199e217b4ff0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.361673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:06.372291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 513/1020] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:06.373307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.400891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:06.438843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1e831-a79f-4573-8152-809b607fd387 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.439138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1e831-a79f-4573-8152-809b607fd387 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.439392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1e831-a79f-4573-8152-809b607fd387 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.441009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 514/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:21:06.446074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.446319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.446628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.464008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:06.465215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:06.480348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6dcf9-9fa4-4ec1-ac5f-bfdc8cfcc4f3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.516266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.516981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.517249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.517534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.518248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] RBAC: Adding query filter params (type=image) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:06.518545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.519946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.529841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 508/1022] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:06.531368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.536205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 515/1023] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:06.560045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.622668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:06.639449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721eeb84-f08e-4106-b75d-633985b97279 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.660461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8f358-b896-4a55-8fe8-221db05d0c13 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.676872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.677765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.678206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.678648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.679644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] RBAC: Adding query filter params (interface=public, service_id=e881530d2380406eab0c7ef4acf207ab, region_id=RegionOne) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:06.680108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.681663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.687536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 509/1024] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:21:06 2022] GET /identity/v3/endpoints?service_id=e881530d2380406eab0c7ef4acf207ab&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:06.692764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:06.693245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.702661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.703589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.703981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.704260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.705467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.706919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.708547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 516/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:06.714156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:06.805774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215b65e-f8f3-4c88-914c-ef35a9d3e1d5 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.813421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f051-e496-434c-9223-ee9edb53f0db None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:06.842337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c763db-3eba-4e60-96dd-defb2cb706c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.843335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c763db-3eba-4e60-96dd-defb2cb706c9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.843609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c763db-3eba-4e60-96dd-defb2cb706c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.843835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c763db-3eba-4e60-96dd-defb2cb706c9 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.844602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c763db-3eba-4e60-96dd-defb2cb706c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.846121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c763db-3eba-4e60-96dd-defb2cb706c9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.852459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ec5fd-5eb2-40a6-96f8-1d842ef7e647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:06.852614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 510/1026] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:21:06 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:21:06.853490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ec5fd-5eb2-40a6-96f8-1d842ef7e647 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:06.853773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ec5fd-5eb2-40a6-96f8-1d842ef7e647 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:06.854048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ec5fd-5eb2-40a6-96f8-1d842ef7e647 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:06.854759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ec5fd-5eb2-40a6-96f8-1d842ef7e647 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:06.856199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ec5fd-5eb2-40a6-96f8-1d842ef7e647 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:06.888558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 517/1027] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:21:06 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:21:07.471724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8200-f5d7-4c92-bc2e-c9c8d862c17c None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:07.472271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8200-f5d7-4c92-bc2e-c9c8d862c17c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:07.472271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8200-f5d7-4c92-bc2e-c9c8d862c17c None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:07.473660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 511/1028] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:07.479245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:07.479658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:07.479778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:07.497491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:07.498688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:07.545955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:07.573320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 518/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:07.597106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8200-f5d7-4c92-bc2e-c9c8d862c17c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:07.616225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:07.702779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8200-f5d7-4c92-bc2e-c9c8d862c17c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:07.737141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:07.737886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:07.738163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:07.738435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:07.749090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:07.757588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:07.758524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:21:07.759380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:07.760662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 512/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jun 30 12:21:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:07.765155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:07.868304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79daa4-514c-4bc3-a9a5-f3930a6602fa None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:07.902861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:07.903906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:07.904391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:07.904895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:07.905889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:07.906410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:07.908179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:07.917023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 519/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:21:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:07.922644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:08.005809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:08.030970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd170fec-566b-4519-9b13-e4d1060f2dd3 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:08.069933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:08.070744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.071025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.071295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.081983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:08.083449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:08.101356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:08.102291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:08.103256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:08.103256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:08.103256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:08.103256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:08.103256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 513/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:08.108512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:08.153253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:08.220125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fadf80-4bce-4712-940e-fa57d399f579 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:08.253780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:08.254452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.254732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.255004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.255724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:08.256084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:08.257431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:08.269744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 520/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:08.275400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:08.379965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d3c8-179e-4080-a7dd-de15db2b6bdb None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:08.406184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:08.415925 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:08.416774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.417030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.417326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.425700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:08.427171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:21:08.442043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:08.443030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:08.444387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 514/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:21:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:08.449969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:08.550293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:08.553360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d39f86-3327-4903-b2ed-23dc8a497f5b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:08.589326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:08.590070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.590357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.590593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.591513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:08.591826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:08.593312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:08.611443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 521/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:21:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:08.617025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:08.721060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c267dd-97cf-46b3-88cb-cf97e752130a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:08.754791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c714632-53cb-491b-b229-b2cb924f53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:08.755441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c714632-53cb-491b-b229-b2cb924f53de None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.755712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c714632-53cb-491b-b229-b2cb924f53de None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.755983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c714632-53cb-491b-b229-b2cb924f53de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.811428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:08.825667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c714632-53cb-491b-b229-b2cb924f53de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:08.827133 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c714632-53cb-491b-b229-b2cb924f53de None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:08.929770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 515/1036] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:08.935489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:08.939660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90218ace-9c43-4888-b7b5-effca2b8307e None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.939931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90218ace-9c43-4888-b7b5-effca2b8307e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.940199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90218ace-9c43-4888-b7b5-effca2b8307e None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.941935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 516/1037] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:21:08.948046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeddde8-8303-4b12-b7e4-3e5ae890123d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:08.948330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeddde8-8303-4b12-b7e4-3e5ae890123d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:08.948610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeddde8-8303-4b12-b7e4-3e5ae890123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:08.967198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7aeddde8-8303-4b12-b7e4-3e5ae890123d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:08.969250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7aeddde8-8303-4b12-b7e4-3e5ae890123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:09.037657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 517/1038] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:09.039141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132ff6-d17f-4caa-bc09-3e40ed444401 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.039535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132ff6-d17f-4caa-bc09-3e40ed444401 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.039800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132ff6-d17f-4caa-bc09-3e40ed444401 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.044662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 518/1039] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:09.046542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.046990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.047393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b30636-f35e-4ba6-8c8b-38b5544ad330 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.047691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.065648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:09.066857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:09.081273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.084783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.085592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.085874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.086230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.096543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.098000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:09.116050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:09.117041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:09.117982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:09.117982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:09.117982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:09.117982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:09.117982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 522/1040] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:09.118271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.131945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.139692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 519/1041] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:09.140518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.225003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd342-b5e3-4492-b3ba-98dcd364c069 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.248588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6f142-cba1-463c-8077-203835ccd3f7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.258096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.258894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.259160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.259426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] PATH_INFO: `/v3/services/image` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.260073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.261467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:09.273211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:09.274252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:09.275101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 523/1042] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:09.276153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.287818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.288488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.288816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.289089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.289820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:09.290126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.291511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.302881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 520/1043] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:09.303679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.315760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.350821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.377210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908139b3-5054-44e7-b52f-622bb6d4c6ae None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.405582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848707a3-a2bd-40b8-b547-609eaccc9790 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.411407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.412238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.412524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.412824 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.413997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.415439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.417124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 524/1044] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:09.417877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.444199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.444893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.445169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.445529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.446278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] RBAC: Adding query filter params (name=image) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:09.446582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.447943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.452887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 521/1045] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:09.453779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.522913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39c919-aa73-4b5b-857b-23eb8529994a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.523397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.553293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd9425-e871-4b24-b9ad-ff5eb17782fe None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.557027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.557809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.558087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.558358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.564998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.566511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:21:09.580890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:09.581991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 525/1046] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:09.583127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.585240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.592487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.593202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.593482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.594117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.595421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.596893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.618002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 522/1047] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:21:09 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:09.619565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.698629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721562c-2526-4dd5-9541-297beb5a28ad None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.733216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c612cdf-e78a-4beb-88d5-37483de84249 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.735009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.735892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.736175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.736479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.737228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] RBAC: Adding query filter params (type=image) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:09.737535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.738944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.744487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 526/1048] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:09.750807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.771670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.784104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.784818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.786219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.786219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.786219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:09.786457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.787895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.803134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 523/1049] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:09.809427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:09.838093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:09.877465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd19506-47b4-4952-a6f7-e23cd80a7703 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.918137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09de7-d1f9-462c-8ecb-5ee2dedca94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.918818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09de7-d1f9-462c-8ecb-5ee2dedca94a None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.919099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09de7-d1f9-462c-8ecb-5ee2dedca94a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.919372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09de7-d1f9-462c-8ecb-5ee2dedca94a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.920103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09de7-d1f9-462c-8ecb-5ee2dedca94a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.921532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09de7-d1f9-462c-8ecb-5ee2dedca94a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.923360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224e00-53c8-4e39-bacc-cbd43ff3d192 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:09.943073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 527/1050] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:21:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:21:09.972528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:09.973250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:09.973514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:09.973777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:09.974617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] RBAC: Adding query filter params (role.id=5907c6a5fd9d42d5985eacd0cf4a6706, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0ca71f32f61e41288cac452e58b4a1dc) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:09.974930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:09.976400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808c3d9-06d7-4689-8a74-a01f207a5d87 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:09.988905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 524/1051] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:09 2022] GET /identity/v3/role_assignments?user.id=0ca71f32f61e41288cac452e58b4a1dc&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=5907c6a5fd9d42d5985eacd0cf4a6706 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:11.729090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3703b-8105-4840-b113-f2ee32d8d2e2 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:11.729649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3703b-8105-4840-b113-f2ee32d8d2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:11.729649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3703b-8105-4840-b113-f2ee32d8d2e2 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:11.730989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 528/1052] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:11.737720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab0d6e-8267-4920-a63d-8d9a7981fdf3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:11.738199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab0d6e-8267-4920-a63d-8d9a7981fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:11.738281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab0d6e-8267-4920-a63d-8d9a7981fdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:11.756374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bcab0d6e-8267-4920-a63d-8d9a7981fdf3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:11.757654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bcab0d6e-8267-4920-a63d-8d9a7981fdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:11.805801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:11.832199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 525/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:11.856592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3703b-8105-4840-b113-f2ee32d8d2e2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:11.966928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3703b-8105-4840-b113-f2ee32d8d2e2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:11.987360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c8ccf-1e65-40dc-8ff4-d115e218ec03 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:11.987549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c8ccf-1e65-40dc-8ff4-d115e218ec03 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:11.987810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c8ccf-1e65-40dc-8ff4-d115e218ec03 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:11.989217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 526/1054] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:11.990881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709df49a-c130-4917-8423-d80920e52918 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:11.991294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:11.991517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709df49a-c130-4917-8423-d80920e52918 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:11.991802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709df49a-c130-4917-8423-d80920e52918 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:11.993233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 527/1055] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:21:11.994348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:11.994629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:11.994889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.005768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f2d3-7e42-4fe2-a322-f57012f47665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.006544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f2d3-7e42-4fe2-a322-f57012f47665 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.006827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f2d3-7e42-4fe2-a322-f57012f47665 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.007100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f2d3-7e42-4fe2-a322-f57012f47665 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.013105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:12.014311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:12.018852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774f2d3-7e42-4fe2-a322-f57012f47665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-6774f2d3-7e42-4fe2-a322-f57012f47665 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:12.026382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:12.027388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:12.028398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 529/1056] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:12.029229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.029383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.029772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.046325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:12.047671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:12.084428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 528/1057] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:12.085301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.117383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 530/1058] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:12.118236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.190264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b147f0-0a99-4d0e-95e9-f5f585b54de0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.199721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.223622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f841-5bc7-4d2a-b2fd-da1c8583fac2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.225237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.225908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.226184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.226448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.227189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:12.227488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.228881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.236955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 529/1059] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:12.237178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.237716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.259989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.260672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.261014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.261284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.329901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.331361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.334116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816e3c-2dc1-4058-8629-87944fa11437 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.367195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.368014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.368294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.368568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.378989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:12.387241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:12.388378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 530/1060] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:12.389572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.420128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.436220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 531/1061] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:12.440862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.454485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.493982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33b0c-1669-47b4-88c1-bfcdda5e570f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.527662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.528533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.528829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.529081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] PATH_INFO: `/v3/users/c6b05b4983e64688a7e5788c7558a342` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.538660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beec5e2-99bb-4137-9165-92802778d506 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.542826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] RBAC: Authorizing `identity:get_user(user_id=c6b05b4983e64688a7e5788c7558a342)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.544347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.546140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 531/1062] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/users/c6b05b4983e64688a7e5788c7558a342 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:12.546894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.576753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.577414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.577688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.577961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.578904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:12.579197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.580571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.588767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 532/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:12.589748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.638098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.650685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544ed5d-bbf7-4477-a985-5f6dc92d1588 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.665076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.684687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.685487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.685761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.686029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.690134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a879c-d0e7-4b96-928d-4dd4742fa47e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.696941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.698810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:12.717098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:12.718021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:12.719326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:12.719326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:12.719326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:12.719326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:12.719326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 532/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:12.719854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.730049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.730707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.730964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.731242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.797897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.799274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.817918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069a0ae-f82c-4863-8cd3-cd91cce2d4d8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:12.852719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:12.853516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:12.853796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:12.854065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:12.862070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.864339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:12.865892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:12.876662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:12.883644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:12.884762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:12.885666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:12.885666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:12.885666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:12.885666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:12.885666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 533/1065] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:12.885666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.907790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 533/1066] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:12.909305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:12.991758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0b1ef-6709-4b7c-9469-745244d402fe None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.014385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a821c-d70d-42e6-9f7c-d1c1448904f8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.026097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.026760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.027028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.027305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.028012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:13.028318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.029721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.041093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 534/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:13.041902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.049439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.050393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.050856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.051407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.052221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:13.052530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.054137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.068063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 534/1068] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:13.074102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.102666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:13.112119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:13.166158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a45b23-1024-4163-b876-67ff8ef37867 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.189966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28806e7-4b2c-4da6-9a09-cf2f98418d09 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.209768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.210657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.210938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.211205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.212423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] RBAC: Authorizing `identity:get_project(project_id=be799c9671fa436799ab083cdad37a09)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.214734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.216648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 535/1069] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/projects/be799c9671fa436799ab083cdad37a09 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:13.222737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.227938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.228780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.229071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.229339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.238134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.239883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:21:13.255811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:13.256941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:13.257980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 535/1070] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:21:13.264814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.342008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370278d8-8e07-4a53-8146-cfe842539666 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.362073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:13.374057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:13.376917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.377579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.377860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.378227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.379155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=be799c9671fa436799ab083cdad37a09, user.id=c6b05b4983e64688a7e5788c7558a342) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:13.379268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.380717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.381677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd383-eb49-457c-813e-dd6184bf52ab None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.386215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 536/1071] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/role_assignments?user.id=c6b05b4983e64688a7e5788c7558a342&scope.project.id=be799c9671fa436799ab083cdad37a09&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:13.419026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.419727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.420002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.420275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.421171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:13.421484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.423157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.445005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 536/1072] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:13.451002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.560297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbd90-db36-4cd3-985d-d59ca796378c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.595408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.596135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.596412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.596717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.607457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.608951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:13.625860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:13.626898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:13.627853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:13.627853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:13.627853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:13.627853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:13.627853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 537/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:13.632966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.735983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc71d4-6712-4d92-8747-f80120dbabb3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.773399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.774075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.774355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.774628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.775355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:13.775661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.777085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.789029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 537/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:13.794850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.833869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:13.898890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056790d8-0f0a-4da3-a325-56c5feb196c8 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:13.933246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:13.934039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:13.934318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:13.934591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:13.941868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:13.943703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:21:13.958292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:13.959182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:13.960067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 538/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:13.965610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:13.981415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:14.070801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684565af-673a-4652-a97a-f0d05d520c2c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:14.105813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:14.106485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:14.106765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:14.107039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:14.107825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:14.108135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:14.109616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f23199-fda5-411b-aeb0-781cd5e5f670 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:14.121986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 538/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:14.128034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:14.227404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:14.234257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3dce-f33f-464b-971a-4c72da5241ed None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:14.269854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12228aed-7e4c-469c-b611-f83d373376ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:14.270749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228aed-7e4c-469c-b611-f83d373376ba None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:14.271046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228aed-7e4c-469c-b611-f83d373376ba None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:14.271319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228aed-7e4c-469c-b611-f83d373376ba None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0ca71f32f61e41288cac452e58b4a1dc/roles/5907c6a5fd9d42d5985eacd0cf4a6706` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:14.273717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12228aed-7e4c-469c-b611-f83d373376ba None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=0ca71f32f61e41288cac452e58b4a1dc, role_id=5907c6a5fd9d42d5985eacd0cf4a6706)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:14.275155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12228aed-7e4c-469c-b611-f83d373376ba None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:14.289418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 539/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:21:14 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0ca71f32f61e41288cac452e58b4a1dc/roles/5907c6a5fd9d42d5985eacd0cf4a6706 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:21:15.380912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0594e5-dbb7-4a87-ab83-aaac69958fa6 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:15.381523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0594e5-dbb7-4a87-ab83-aaac69958fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:15.381523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0594e5-dbb7-4a87-ab83-aaac69958fa6 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:15.382881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 539/1078] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:15.389113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:15.389668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:15.389668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:15.407426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:15.408663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:15.477536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 540/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:15.500444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0594e5-dbb7-4a87-ab83-aaac69958fa6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:15.572062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:15.604212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0594e5-dbb7-4a87-ab83-aaac69958fa6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:15.639517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:15.640307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:15.640589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:15.641178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:15.651784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:15.659521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:15.660483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:15.661383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 540/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:15.667486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:15.752062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:15.769882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730838-4fab-47ff-a1f3-241e8cac68a3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:15.804410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:15.805141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:15.805421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:15.805689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:15.806438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:15.806743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:15.808116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:15.816532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 541/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:15.823333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:15.928372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11e327-6c9f-4ca7-aa24-86b8ae6d793d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:15.959107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:15.964923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:15.965804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:15.966081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:15.966354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] PATH_INFO: `/v3/users/c6b05b4983e64688a7e5788c7558a342` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:15.967560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] RBAC: Authorizing `identity:get_user(user_id=c6b05b4983e64688a7e5788c7558a342)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:15.969137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:15.970813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 541/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:15 2022] GET /identity/v3/users/c6b05b4983e64688a7e5788c7558a342 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:15.977034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:16.079555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854337d3-a03f-4863-aa95-eccc907b0b63 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:16.114549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339934b-0bd9-4237-b3a3-24ebbc840430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:16.115413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339934b-0bd9-4237-b3a3-24ebbc840430 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.115697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339934b-0bd9-4237-b3a3-24ebbc840430 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.115969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339934b-0bd9-4237-b3a3-24ebbc840430 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.117237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339934b-0bd9-4237-b3a3-24ebbc840430 None admin] RBAC: Authorizing `identity:get_project(project_id=be799c9671fa436799ab083cdad37a09)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:16.118692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339934b-0bd9-4237-b3a3-24ebbc840430 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:16.120412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 542/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:15 2022] GET /identity/v3/projects/be799c9671fa436799ab083cdad37a09 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:16.126820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:16.232152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2851-9af9-4f85-b2f8-cd692fd4cd45 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:16.266992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d32ed-7a44-4580-af13-bc55002e4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:16.267881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d32ed-7a44-4580-af13-bc55002e4b40 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.268161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d32ed-7a44-4580-af13-bc55002e4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.268433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d32ed-7a44-4580-af13-bc55002e4b40 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09/users/c6b05b4983e64688a7e5788c7558a342/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.270549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d32ed-7a44-4580-af13-bc55002e4b40 None admin] RBAC: Authorizing `identity:create_grant(project_id=be799c9671fa436799ab083cdad37a09, user_id=c6b05b4983e64688a7e5788c7558a342, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:16.271985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d32ed-7a44-4580-af13-bc55002e4b40 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:16.288470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 542/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:21:16 2022] PUT /identity/v3/projects/be799c9671fa436799ab083cdad37a09/users/c6b05b4983e64688a7e5788c7558a342/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:21:16.315545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a910f6-bb8c-4dc3-943b-54db6bc8634a None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.315831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a910f6-bb8c-4dc3-943b-54db6bc8634a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.316108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a910f6-bb8c-4dc3-943b-54db6bc8634a None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.317598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 543/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:16.324273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.324537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.324873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.344056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:16.345690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:16.408182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:16.421099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 543/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:16.445929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a910f6-bb8c-4dc3-943b-54db6bc8634a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:16.465148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:16.550327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a910f6-bb8c-4dc3-943b-54db6bc8634a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:16.585062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:16.585875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.586142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.586420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.597026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:16.604942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:16.605872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:16.606701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 544/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jun 30 12:21:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:16.612535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:16.712736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898876-7970-4ea0-bdc2-e2e7375018d1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:16.748328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:16.749026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.749284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.749554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.750487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:16.750965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:16.752338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:16.760438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 544/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:21:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:16.766759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:16.846398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:16.869538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89790156-5e5e-492b-ab57-e119aa498f15 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:16.905455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:16.906249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:16.906606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:16.907359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:16.917456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:16.918902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:16.936715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:16.937607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:16.939022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:16.939022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:16.939022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:16.939022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:16.939022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 545/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:16.944845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:16.997821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:17.046432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22910d-ec70-4409-9714-42d033da7539 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:17.081948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:17.082602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:17.082876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:17.083140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:17.083851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:17.084140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:17.085545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:17.097493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 545/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:17.103199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:17.207503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6007e-a881-47a8-96d1-f1611f514b7e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:17.233816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:17.242900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:17.243709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:17.243987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:17.244239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:17.251510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:17.253020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:21:17.267423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:17.268382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:17.269357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 546/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jun 30 12:21:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:17.274615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:17.376616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c27a0d-7485-468c-8c5a-16d5060e9abb None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:17.392306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:17.416048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:17.416743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:17.417037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:17.417310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:17.418150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:17.418462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:17.419907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:17.438544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 546/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jun 30 12:21:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:17.444928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:17.549600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f98c0e-838f-4f28-9b06-237260cd69db None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:17.586109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:17.586760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:17.587041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:17.587317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:17.641100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:17.654940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:17.656507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:17.756989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 547/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:17.762902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:17.868523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a919-d36e-4551-a866-ad1535bc0813 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:17.905063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:17.905860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:17.906139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:17.906411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:17.916971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:17.918417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:17.935969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:17.936937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:17.937906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:21:17.937906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:17.937906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:21:17.937906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:17.937906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 547/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:17.943739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.033441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:18.046381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855582-dbee-45b0-8a5e-34c2fab2cc96 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.083580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.084240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.084510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.084811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.085524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:18.085832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:18.087173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:18.099137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 548/1095] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:18.104930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.130801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:18.216649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005253-bd92-4f7a-827c-f7e3f224d5d6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.252784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.253581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.253854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.254121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.261042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:18.262756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-51ced82c-db14-40a0-bf4b-fe6848b88efe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:21:18.276995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:18.277934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:18.278859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 548/1096] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:18.284917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.297555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3663a-1c12-4032-aa08-561f4e26177c None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.298318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3663a-1c12-4032-aa08-561f4e26177c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.298596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3663a-1c12-4032-aa08-561f4e26177c None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.299941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 549/1097] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:21:18.304930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.305206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.305474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.323992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:18.325217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:18.387965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050ed3b-d42b-43a5-8c6c-9b6ef9dbd18e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.392369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 550/1098] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:18.415629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.423066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.423736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.424016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.424279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.425085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:18.425414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:18.426835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:18.429279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:18.440221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 549/1099] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:18.446594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.457646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:18.514484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb462fa-a662-4e1f-9870-b61fb6492e7d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.546482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bd1a4-f64b-482b-8cb4-a4a948077b4f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.550023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.550836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.551118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.551386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.561910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:18.569939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:18.571048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:18.572343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 551/1100] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:21:18.577292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.587246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.587908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.588184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.588455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.589331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] RBAC: Adding query filter params (role.id=5907c6a5fd9d42d5985eacd0cf4a6706, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0ca71f32f61e41288cac452e58b4a1dc) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:18.589628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:18.591067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:18.597690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 550/1101] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/role_assignments?user.id=0ca71f32f61e41288cac452e58b4a1dc&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=5907c6a5fd9d42d5985eacd0cf4a6706 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:18.656096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:18.690909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2c44-20c5-4fcd-b1ba-e91bfe5c36a2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.728796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.729482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.729759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.730032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.730795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:18.731098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:18.732486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:18.745887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 552/1102] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:21:18.752723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:18.827125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:18.859471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f21a2-2f0f-40c9-a1e8-2cae381d1cf0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:18.895496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:18.896396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:18.896664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:18.896959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] PATH_INFO: `/v3/users/c6b05b4983e64688a7e5788c7558a342` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:18.898206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] RBAC: Authorizing `identity:get_user(user_id=c6b05b4983e64688a7e5788c7558a342)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:18.899655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:18.901547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 551/1103] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/users/c6b05b4983e64688a7e5788c7558a342 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:18.907618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:19.015672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37108cd2-4a20-46b2-8251-e1c9ff98c98e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:19.052750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:19.053408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:19.053692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:19.053956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:19.056745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:19.125269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:19.126677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:19.229838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 553/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:19.236194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:19.346161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b052ace-8298-416f-98d0-602d6c5fec98 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:19.380114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750a23-f67b-4f25-acc8-8f1cffc01ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:19.380938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750a23-f67b-4f25-acc8-8f1cffc01ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:19.381221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750a23-f67b-4f25-acc8-8f1cffc01ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:19.381497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750a23-f67b-4f25-acc8-8f1cffc01ad1 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:19.382729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77750a23-f67b-4f25-acc8-8f1cffc01ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=be799c9671fa436799ab083cdad37a09)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:19.384177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77750a23-f67b-4f25-acc8-8f1cffc01ad1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:19.385951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 552/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:19 2022] GET /identity/v3/projects/be799c9671fa436799ab083cdad37a09 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:19.392805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:19.430503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:19.499145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482cb76-1ecd-4b06-817b-460612c102f6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:19.533310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:19.533966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:19.534249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:19.534518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:19.535348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=be799c9671fa436799ab083cdad37a09, user.id=c6b05b4983e64688a7e5788c7558a342) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:19.535659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:19.537124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8846f-0c62-4205-be7d-70a172403718 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:19.543160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 554/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:19 2022] GET /identity/v3/role_assignments?user.id=c6b05b4983e64688a7e5788c7558a342&scope.project.id=be799c9671fa436799ab083cdad37a09&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:21.678224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fede9ce-4547-4433-82e8-f8994c279692 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:21.678900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fede9ce-4547-4433-82e8-f8994c279692 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:21.679117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fede9ce-4547-4433-82e8-f8994c279692 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:21.680511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 553/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:21.686949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:21.687458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:21.687458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:21.708397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:21.709654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:21.785966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 555/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:21.815508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fede9ce-4547-4433-82e8-f8994c279692 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:21.828689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:21.945614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fede9ce-4547-4433-82e8-f8994c279692 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:21.989334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e18c-669d-477f-baa1-4bc942b1ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:21.990150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e18c-669d-477f-baa1-4bc942b1ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:21.990429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e18c-669d-477f-baa1-4bc942b1ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:21.990700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e18c-669d-477f-baa1-4bc942b1ffa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:21.991976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10e18c-669d-477f-baa1-4bc942b1ffa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:21.993527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10e18c-669d-477f-baa1-4bc942b1ffa4 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:21.995200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 554/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:22.001533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:22.033362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:22.106383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242010-55dc-4bd1-a778-8afe54bedad9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:22.145003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74725bf-5bce-4764-90de-df4af305d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:22.145663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74725bf-5bce-4764-90de-df4af305d57a None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:22.145938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74725bf-5bce-4764-90de-df4af305d57a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:22.146207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74725bf-5bce-4764-90de-df4af305d57a None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:22.146893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74725bf-5bce-4764-90de-df4af305d57a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:22.148319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74725bf-5bce-4764-90de-df4af305d57a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:22.171175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 556/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:21:22 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:24.211340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d419d83-98d7-4e6a-aefb-3992d9305995 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:24.211838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d419d83-98d7-4e6a-aefb-3992d9305995 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:24.211838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d419d83-98d7-4e6a-aefb-3992d9305995 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:24.213288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 555/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:24.218958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:24.219419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:24.219528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:24.236981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:24.238165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:24.307163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 557/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:24.341387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d419d83-98d7-4e6a-aefb-3992d9305995 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:24.438835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:24.464798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d419d83-98d7-4e6a-aefb-3992d9305995 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:24.500197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:24.501134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:24.501418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:24.501691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] PATH_INFO: `/v3/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:24.503298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] RBAC: Authorizing `identity:get_role(role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:24.504657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:24.506043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 556/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:24 2022] GET /identity/v3/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:24.512892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:24.529468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:24.617467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dd2a1-9f40-4ae1-bbd6-580b01e89d67 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:24.652787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:24.653652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:24.653932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:24.654206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] PATH_INFO: `/v3/users/75d5e6ffda2d4d84b26b70bb5e9fcf15` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:24.668016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] RBAC: Authorizing `identity:get_user(user_id=75d5e6ffda2d4d84b26b70bb5e9fcf15)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:24.669501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:24.671157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 558/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:24 2022] GET /identity/v3/users/75d5e6ffda2d4d84b26b70bb5e9fcf15 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:24.677271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:24.794869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c28bd8-0f72-46e6-976b-7200cb001ad7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:24.830793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:24.831436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:24.831717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:24.831987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:24.846854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:24.904330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:24.905825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:25.006447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 557/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:25.012599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:25.086079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:25.120988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a9607-d329-454c-983a-3bc5818e291a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:25.157885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a5c5e-5176-4754-9725-82401929dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:25.158735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a5c5e-5176-4754-9725-82401929dc53 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:25.159013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a5c5e-5176-4754-9725-82401929dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:25.159279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a5c5e-5176-4754-9725-82401929dc53 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:25.160501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a5c5e-5176-4754-9725-82401929dc53 None admin] RBAC: Authorizing `identity:get_project(project_id=be799c9671fa436799ab083cdad37a09)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:25.161972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a5c5e-5176-4754-9725-82401929dc53 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:25.163673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 559/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:25 2022] GET /identity/v3/projects/be799c9671fa436799ab083cdad37a09 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:25.170462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:25.220876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:25.276731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c155bf-8e0f-49fc-ad60-49dca3166b54 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:25.312573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:25.313737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:25.314014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:25.314283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:25.315111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] RBAC: Adding query filter params (role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=be799c9671fa436799ab083cdad37a09, user.id=75d5e6ffda2d4d84b26b70bb5e9fcf15) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:25.315419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:25.316867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a20ea5-b0ac-440b-838e-ed1cb4522e86 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:25.322714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 558/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:25 2022] GET /identity/v3/role_assignments?user.id=75d5e6ffda2d4d84b26b70bb5e9fcf15&scope.project.id=be799c9671fa436799ab083cdad37a09&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:27.313536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10bc1-c8dc-407d-a543-224b938741ef None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:27.314080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10bc1-c8dc-407d-a543-224b938741ef None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:27.314080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10bc1-c8dc-407d-a543-224b938741ef None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:27.316230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 560/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:27.325624 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:27.326032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:27.326240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:27.351536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:27.352764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:27.433636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 559/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:27.457149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10bc1-c8dc-407d-a543-224b938741ef None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:27.568270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10bc1-c8dc-407d-a543-224b938741ef None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:27.603598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:27.604436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:27.604743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:27.605005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] PATH_INFO: `/v3/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:27.606508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] RBAC: Authorizing `identity:get_role(role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:27.607869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:27.609268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 561/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:27 2022] GET /identity/v3/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:27.616002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:27.667855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:27.732124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae98bb-e2c1-4285-a850-052473ccad47 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:27.779930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:27.781321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:27.782254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:27.782254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] PATH_INFO: `/v3/users/75d5e6ffda2d4d84b26b70bb5e9fcf15` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:27.784187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] RBAC: Authorizing `identity:get_user(user_id=75d5e6ffda2d4d84b26b70bb5e9fcf15)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:27.786566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:27.789289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 560/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:27 2022] GET /identity/v3/users/75d5e6ffda2d4d84b26b70bb5e9fcf15 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:27.798420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:27.809914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:27.902937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40a13-16ab-454c-b119-800607b2a9e3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:27.937898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea1e6-2bd2-43a3-8f78-cd1d8141fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:27.938740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea1e6-2bd2-43a3-8f78-cd1d8141fff0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:27.939032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea1e6-2bd2-43a3-8f78-cd1d8141fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:27.939304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea1e6-2bd2-43a3-8f78-cd1d8141fff0 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:27.940585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adea1e6-2bd2-43a3-8f78-cd1d8141fff0 None admin] RBAC: Authorizing `identity:get_project(project_id=be799c9671fa436799ab083cdad37a09)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:27.942157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adea1e6-2bd2-43a3-8f78-cd1d8141fff0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:27.944232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 562/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:27 2022] GET /identity/v3/projects/be799c9671fa436799ab083cdad37a09 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:27.951191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:28.056531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878b90-8ea9-45c5-be3b-2fe1a5cf3f2e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:28.064036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:28.098330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd293598-a8f0-46e3-ad2d-697aa5c1b75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:28.099226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd293598-a8f0-46e3-ad2d-697aa5c1b75c None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:28.099504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd293598-a8f0-46e3-ad2d-697aa5c1b75c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:28.099781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd293598-a8f0-46e3-ad2d-697aa5c1b75c None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09/users/75d5e6ffda2d4d84b26b70bb5e9fcf15/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:28.102229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd293598-a8f0-46e3-ad2d-697aa5c1b75c None admin] RBAC: Authorizing `identity:create_grant(project_id=be799c9671fa436799ab083cdad37a09, user_id=75d5e6ffda2d4d84b26b70bb5e9fcf15, role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:28.103710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd293598-a8f0-46e3-ad2d-697aa5c1b75c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:28.118360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 561/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:21:27 2022] PUT /identity/v3/projects/be799c9671fa436799ab083cdad37a09/users/75d5e6ffda2d4d84b26b70bb5e9fcf15/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:21:30.098957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68822e5-4817-4176-a647-83b299c8e9f2 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:30.099523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68822e5-4817-4176-a647-83b299c8e9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:30.099523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68822e5-4817-4176-a647-83b299c8e9f2 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:30.100922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 563/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:30.107285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:30.108021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:30.108314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:30.136190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:30.137840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:30.217258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 562/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:30.254129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68822e5-4817-4176-a647-83b299c8e9f2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:30.336597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:30.362326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68822e5-4817-4176-a647-83b299c8e9f2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:30.401178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:30.402032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:30.402313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:30.402586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] PATH_INFO: `/v3/roles/e6c1b1ccb0614e1ab27b89016f60f0c2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:30.404182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] RBAC: Authorizing `identity:get_role(role_id=e6c1b1ccb0614e1ab27b89016f60f0c2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:30.405592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:30.407173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 564/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:30 2022] GET /identity/v3/roles/e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:30.413252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:30.517607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:30.525577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c35e09-eb19-4bd3-b685-67c5176589f0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:30.564163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:30.565073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:30.565333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:30.565601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] PATH_INFO: `/v3/users/75d5e6ffda2d4d84b26b70bb5e9fcf15` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:30.566809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] RBAC: Authorizing `identity:get_user(user_id=75d5e6ffda2d4d84b26b70bb5e9fcf15)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:30.568597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:30.570251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 563/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:30 2022] GET /identity/v3/users/75d5e6ffda2d4d84b26b70bb5e9fcf15 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:30.576278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:30.680224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cfb11-ab79-4ef2-a9c8-58ca8cdd379e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:30.717757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:30.718404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:30.718720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:30.718999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:30.735193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:30.799170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:30.800614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:30.913652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 565/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:30.922215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:31.037525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff73a7-d5fc-44c3-8ce2-1da65254aadf None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:31.076489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20238ad2-5b27-4aba-86b4-177ca4684978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:31.077914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20238ad2-5b27-4aba-86b4-177ca4684978 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:31.078269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20238ad2-5b27-4aba-86b4-177ca4684978 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:31.078541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20238ad2-5b27-4aba-86b4-177ca4684978 None admin] PATH_INFO: `/v3/projects/be799c9671fa436799ab083cdad37a09` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:31.079848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20238ad2-5b27-4aba-86b4-177ca4684978 None admin] RBAC: Authorizing `identity:get_project(project_id=be799c9671fa436799ab083cdad37a09)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:31.081341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20238ad2-5b27-4aba-86b4-177ca4684978 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:31.083051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 564/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:30 2022] GET /identity/v3/projects/be799c9671fa436799ab083cdad37a09 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:31.089532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:31.151872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:31.212487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c5e67-b930-46ab-a788-1c87d2d1ad6d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:31.251077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:31.251729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:31.252008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:31.252278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:31.253144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] RBAC: Adding query filter params (role.id=e6c1b1ccb0614e1ab27b89016f60f0c2, scope.project.id=be799c9671fa436799ab083cdad37a09, user.id=75d5e6ffda2d4d84b26b70bb5e9fcf15) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:31.253457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:31.254869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101b69-82e8-47ca-b1e3-545b8fc73df3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:31.260914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 566/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:31 2022] GET /identity/v3/role_assignments?user.id=75d5e6ffda2d4d84b26b70bb5e9fcf15&scope.project.id=be799c9671fa436799ab083cdad37a09&role.id=e6c1b1ccb0614e1ab27b89016f60f0c2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:33.320032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4134109-b146-40bd-8782-f4a7337b762b None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:33.320655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4134109-b146-40bd-8782-f4a7337b762b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:33.320655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4134109-b146-40bd-8782-f4a7337b762b None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:33.322011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 565/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:33.328423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add645-536d-4174-97fb-b8268aa81474 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:33.329138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add645-536d-4174-97fb-b8268aa81474 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:33.329367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add645-536d-4174-97fb-b8268aa81474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:33.353257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-11add645-536d-4174-97fb-b8268aa81474 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:33.354445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-11add645-536d-4174-97fb-b8268aa81474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:33.428125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 567/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:33.452618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4134109-b146-40bd-8782-f4a7337b762b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:33.488096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:33.578804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4134109-b146-40bd-8782-f4a7337b762b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:33.617472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d97b0-899d-4927-be39-6ff64f40b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:33.618269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d97b0-899d-4927-be39-6ff64f40b6bf None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:33.618549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d97b0-899d-4927-be39-6ff64f40b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:33.618820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d97b0-899d-4927-be39-6ff64f40b6bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:33.620126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d97b0-899d-4927-be39-6ff64f40b6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:33.621622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d97b0-899d-4927-be39-6ff64f40b6bf None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:33.623253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 566/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:33.630258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add645-536d-4174-97fb-b8268aa81474 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:33.673575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:33.745806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add645-536d-4174-97fb-b8268aa81474 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:33.787599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816f6e5-5b21-4a13-ab7d-7af6def9c549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:33.788577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816f6e5-5b21-4a13-ab7d-7af6def9c549 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:33.788991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816f6e5-5b21-4a13-ab7d-7af6def9c549 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:33.789360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816f6e5-5b21-4a13-ab7d-7af6def9c549 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:33.790434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816f6e5-5b21-4a13-ab7d-7af6def9c549 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:33.792727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816f6e5-5b21-4a13-ab7d-7af6def9c549 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:33.826989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 568/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:21:33 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:35.866288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03da2f-f9ee-457d-8434-8f4d0772c19b None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:35.867196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03da2f-f9ee-457d-8434-8f4d0772c19b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:35.867196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03da2f-f9ee-457d-8434-8f4d0772c19b None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:35.869303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 567/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:35.878711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:35.879449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:35.879604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:35.902550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:35.904325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:35.986746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 569/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:36.023581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03da2f-f9ee-457d-8434-8f4d0772c19b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:36.128246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:36.132579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03da2f-f9ee-457d-8434-8f4d0772c19b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:36.176789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384fe92-335b-4db7-b1aa-a5c238401d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:36.178012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384fe92-335b-4db7-b1aa-a5c238401d1b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:36.178449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384fe92-335b-4db7-b1aa-a5c238401d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:36.178873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384fe92-335b-4db7-b1aa-a5c238401d1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:36.181090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384fe92-335b-4db7-b1aa-a5c238401d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:36.183494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384fe92-335b-4db7-b1aa-a5c238401d1b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:36.186225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 568/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:21:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:36.195520 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:36.219266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:36.301056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a10f1-269f-4b60-9c42-c281e3d8fa9a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:36.347748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59e02d-940c-4403-ba22-86e0e2676085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:36.348416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59e02d-940c-4403-ba22-86e0e2676085 None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:36.348735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59e02d-940c-4403-ba22-86e0e2676085 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:36.349003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59e02d-940c-4403-ba22-86e0e2676085 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:36.349705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59e02d-940c-4403-ba22-86e0e2676085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:36.351146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59e02d-940c-4403-ba22-86e0e2676085 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:36.378318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 570/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:21:36 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:38.486382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9e64d-1ce1-41f6-8b7b-b6ec5a40042b None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:38.487279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9e64d-1ce1-41f6-8b7b-b6ec5a40042b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:38.487279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9e64d-1ce1-41f6-8b7b-b6ec5a40042b None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:38.489513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 569/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:38.497401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:38.497836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:38.497949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:38.517574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:38.518779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:38.600843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 571/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:21:38.634576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9e64d-1ce1-41f6-8b7b-b6ec5a40042b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:38.755002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9e64d-1ce1-41f6-8b7b-b6ec5a40042b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:38.793051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:38.793828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:38.794101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:38.794368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:38.805569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:38.813632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:38.814759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:38.815720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 570/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:38.821374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:38.830038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:38.956323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516964-d4c5-4233-804a-0db3f9fc16d0 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:38.992367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:38.993068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:38.993350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:38.993625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:38.994376 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:38.994681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:38.996064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:39.004315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 572/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:39.013747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:39.045481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:39.123104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2651b-f33c-475c-863a-bf8f1910e9ff None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:39.157912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:39.158777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:39.159054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:39.159323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] PATH_INFO: `/v3/users/b5caa156742a4be286a6c33054a3cca8` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:39.179805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] RBAC: Authorizing `identity:get_user(user_id=b5caa156742a4be286a6c33054a3cca8)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:39.182066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:39.184732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 571/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:39 2022] GET /identity/v3/users/b5caa156742a4be286a6c33054a3cca8 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:39.191802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:39.263326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:39.299290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5500053-801f-4a65-bb54-1fdbf4a4fac1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:39.335642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:39.336293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:39.336577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:39.336883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:39.407127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:39.408582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:39.489667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:39.536120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 573/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:39.542201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:39.633102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:39.649573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deff2cc-c7da-4c06-a177-585a31d899ce None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:39.687571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55603b-c0e5-4fe0-b7f7-8de0008e31d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:39.688155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55603b-c0e5-4fe0-b7f7-8de0008e31d1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:39.688434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55603b-c0e5-4fe0-b7f7-8de0008e31d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:39.688737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55603b-c0e5-4fe0-b7f7-8de0008e31d1 None admin] PATH_INFO: `/v3/projects/f469b65d5b404211ba30bf443e23820d` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:39.689971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55603b-c0e5-4fe0-b7f7-8de0008e31d1 None admin] RBAC: Authorizing `identity:get_project(project_id=f469b65d5b404211ba30bf443e23820d)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:39.691410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55603b-c0e5-4fe0-b7f7-8de0008e31d1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:39.693456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 572/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:39 2022] GET /identity/v3/projects/f469b65d5b404211ba30bf443e23820d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:39.699845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:39.800508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ad90-c102-4f26-8730-0f56a90de2d8 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:39.836085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:39.836761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:39.837045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:39.837318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:39.838150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=f469b65d5b404211ba30bf443e23820d, user.id=b5caa156742a4be286a6c33054a3cca8) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:39.838461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:39.839899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26f4ef-e7f0-4f67-b5dc-61bfb7756e42 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:39.846129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 574/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:39 2022] GET /identity/v3/role_assignments?user.id=b5caa156742a4be286a6c33054a3cca8&scope.project.id=f469b65d5b404211ba30bf443e23820d&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:41.832024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94687558-b231-4ec7-8322-f823c85352e4 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:41.832587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94687558-b231-4ec7-8322-f823c85352e4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:41.832587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94687558-b231-4ec7-8322-f823c85352e4 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:41.833977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 573/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:41.840365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:41.841103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:41.841281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:41.865710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:41.866910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:41.895637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:41.945705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 575/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:41.972309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94687558-b231-4ec7-8322-f823c85352e4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:42.082822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94687558-b231-4ec7-8322-f823c85352e4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:42.117523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:42.118308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:42.118588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:42.118858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:42.129606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:42.137194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:42.138239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:42.139741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 574/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:42.148168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:42.272103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d8fa-f49d-4f7a-a462-6ba8b0ae391e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:42.314199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:42.314867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:42.315139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:42.315406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:42.316150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:42.316459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:42.318208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:42.326325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 576/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:42.332658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:42.354923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:42.443832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf71b9-7f16-49a8-9efa-0f32486b5255 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:42.478703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:42.479574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:42.479855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:42.480126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] PATH_INFO: `/v3/users/b5caa156742a4be286a6c33054a3cca8` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:42.481487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] RBAC: Authorizing `identity:get_user(user_id=b5caa156742a4be286a6c33054a3cca8)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:42.482932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:42.484659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 575/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:42 2022] GET /identity/v3/users/b5caa156742a4be286a6c33054a3cca8 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:42.490960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:42.520002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:42.596545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9fe3-f187-478c-a6ff-ff21798018b4 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:42.634531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31358900-7b27-46f6-96d6-0db764d44db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:42.635328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31358900-7b27-46f6-96d6-0db764d44db3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:42.635605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31358900-7b27-46f6-96d6-0db764d44db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:42.635875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31358900-7b27-46f6-96d6-0db764d44db3 None admin] PATH_INFO: `/v3/projects/f469b65d5b404211ba30bf443e23820d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:42.637209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31358900-7b27-46f6-96d6-0db764d44db3 None admin] RBAC: Authorizing `identity:get_project(project_id=f469b65d5b404211ba30bf443e23820d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:42.638644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31358900-7b27-46f6-96d6-0db764d44db3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:42.640460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 577/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:42 2022] GET /identity/v3/projects/f469b65d5b404211ba30bf443e23820d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:42.647197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:42.746308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:42.773655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808f39a-684f-411e-a147-e8a63f5b5192 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:42.809381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc893d-d42e-4f39-a173-fed77ce4d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:42.810263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc893d-d42e-4f39-a173-fed77ce4d850 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:42.810513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc893d-d42e-4f39-a173-fed77ce4d850 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:42.810810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc893d-d42e-4f39-a173-fed77ce4d850 None admin] PATH_INFO: `/v3/projects/f469b65d5b404211ba30bf443e23820d/users/b5caa156742a4be286a6c33054a3cca8/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:42.813150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc893d-d42e-4f39-a173-fed77ce4d850 None admin] RBAC: Authorizing `identity:create_grant(project_id=f469b65d5b404211ba30bf443e23820d, user_id=b5caa156742a4be286a6c33054a3cca8, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:42.814590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc893d-d42e-4f39-a173-fed77ce4d850 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:42.829396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 576/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:21:42 2022] PUT /identity/v3/projects/f469b65d5b404211ba30bf443e23820d/users/b5caa156742a4be286a6c33054a3cca8/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:21:44.822282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884b70-ea48-487d-b567-126a711ca967 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:44.823260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884b70-ea48-487d-b567-126a711ca967 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:44.823260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884b70-ea48-487d-b567-126a711ca967 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:44.825446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 578/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:44.831186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:44.831605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:44.831715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:44.850114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:44.851303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:44.920223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 577/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:44.945104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82884b70-ea48-487d-b567-126a711ca967 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:45.044273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:45.055555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82884b70-ea48-487d-b567-126a711ca967 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:45.091534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:45.092262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:45.092535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:45.092921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:45.105162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:45.112707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:45.113762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:45.114715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 579/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:45.120407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:45.199268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:45.234597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87168d-640f-47f6-b63a-aec3c3805bdb None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:45.275853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:45.276518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:45.276818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:45.277088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:45.277834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:45.278135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:45.279499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:45.287871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 578/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:45.294114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:45.409481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eea5b-32b6-47d9-b1a3-423aacd3d73c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:45.445154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:45.446022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:45.446295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:45.446565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] PATH_INFO: `/v3/users/b5caa156742a4be286a6c33054a3cca8` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:45.447785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] RBAC: Authorizing `identity:get_user(user_id=b5caa156742a4be286a6c33054a3cca8)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:45.449260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:45.450888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 580/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:45 2022] GET /identity/v3/users/b5caa156742a4be286a6c33054a3cca8 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:45.457246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:45.564427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d973db-5647-41c0-9467-c8920310a709 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:45.599547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:45.600176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:45.600451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:45.600749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:45.603627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:45.669188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:45.670580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:45.790957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 579/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:45.798253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:45.827599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:45.908995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ad383-c4cd-4801-9ccd-a4736e857384 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:45.945181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14ba7-932a-478a-bc84-f5f861d9a103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:45.946038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14ba7-932a-478a-bc84-f5f861d9a103 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:45.946311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14ba7-932a-478a-bc84-f5f861d9a103 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:45.946583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14ba7-932a-478a-bc84-f5f861d9a103 None admin] PATH_INFO: `/v3/projects/f469b65d5b404211ba30bf443e23820d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:45.947893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba14ba7-932a-478a-bc84-f5f861d9a103 None admin] RBAC: Authorizing `identity:get_project(project_id=f469b65d5b404211ba30bf443e23820d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:45.949401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba14ba7-932a-478a-bc84-f5f861d9a103 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:45.951101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 581/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:45 2022] GET /identity/v3/projects/f469b65d5b404211ba30bf443e23820d => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:45.957766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:45.994516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:46.065776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104dc20-692b-4493-ba74-4d0302568b8c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:46.101057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:46.101720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:46.101998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:46.102278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:46.103113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=f469b65d5b404211ba30bf443e23820d, user.id=b5caa156742a4be286a6c33054a3cca8) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:46.103431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:46.104952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5cf42-f30f-4659-a503-ba7b49c80ae1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:46.110910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 580/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:45 2022] GET /identity/v3/role_assignments?user.id=b5caa156742a4be286a6c33054a3cca8&scope.project.id=f469b65d5b404211ba30bf443e23820d&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:48.121565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46b1bf-712c-4659-b478-13bb5322e18c None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:48.122149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46b1bf-712c-4659-b478-13bb5322e18c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:48.122149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46b1bf-712c-4659-b478-13bb5322e18c None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:48.123509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 582/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:48.130071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:48.130750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:48.130923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:48.158399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:48.159591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:48.231352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 581/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:48.254587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46b1bf-712c-4659-b478-13bb5322e18c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:48.344654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:48.361187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46b1bf-712c-4659-b478-13bb5322e18c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:48.398292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:48.398870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:48.399147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:48.399423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:48.474255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:48.476467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:48.569751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:48.583467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 583/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:48.593218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:48.629111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:48.710046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de717e78-4bc7-4e1d-8763-a93f517d51c9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:48.753212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:48.754488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:48.754962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:48.755420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:48.766813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:48.768282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-4480f955-af08-43ec-88ad-ad99337762d6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:48.783429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:48.784914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:21:48.786032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 582/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:21:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:48.790798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:48.909402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9641f1-f021-4f2c-9ced-4cd93433194e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:48.946944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:48.947596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:48.947874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:48.948148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:48.948888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:48.949197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:48.950577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90493281-d4f2-4334-803b-76d16c3e1ac6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:48.957811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 584/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jun 30 12:21:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:50.965524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f63a7-46fe-4f41-b410-43c2925904b9 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:50.966091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f63a7-46fe-4f41-b410-43c2925904b9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:50.966091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f63a7-46fe-4f41-b410-43c2925904b9 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:50.967476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 583/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:50.973947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e153b-dcc2-440f-9081-f329bcfa3f72 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:50.974634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e153b-dcc2-440f-9081-f329bcfa3f72 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:50.974773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e153b-dcc2-440f-9081-f329bcfa3f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:50.996658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-673e153b-dcc2-440f-9081-f329bcfa3f72 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:50.997865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-673e153b-dcc2-440f-9081-f329bcfa3f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:51.048251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:51.072911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 585/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:51.097129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f63a7-46fe-4f41-b410-43c2925904b9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:51.210109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:51.225222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f63a7-46fe-4f41-b410-43c2925904b9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:51.262227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc66b00-35fb-4aa3-9b39-67f8af7ca83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:51.262890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc66b00-35fb-4aa3-9b39-67f8af7ca83c None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:51.263172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc66b00-35fb-4aa3-9b39-67f8af7ca83c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:51.263446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc66b00-35fb-4aa3-9b39-67f8af7ca83c None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:51.264208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc66b00-35fb-4aa3-9b39-67f8af7ca83c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:51.265615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc66b00-35fb-4aa3-9b39-67f8af7ca83c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:51.283064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 584/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu Jun 30 12:21:51 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:21:53.330296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7552fee-6a7e-431b-b362-5736d38e7876 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:53.331212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7552fee-6a7e-431b-b362-5736d38e7876 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:53.331212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7552fee-6a7e-431b-b362-5736d38e7876 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:53.333518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 586/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:53.339894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:53.340365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:53.340466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:53.360390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:53.361644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:53.432128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 585/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:53.457882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7552fee-6a7e-431b-b362-5736d38e7876 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:53.571773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7552fee-6a7e-431b-b362-5736d38e7876 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:53.606230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10c050-3db1-4ea1-89aa-8ec324cf1882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:53.607081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10c050-3db1-4ea1-89aa-8ec324cf1882 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:53.607359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10c050-3db1-4ea1-89aa-8ec324cf1882 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:53.607631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10c050-3db1-4ea1-89aa-8ec324cf1882 None admin] PATH_INFO: `/v3/domains/d8cf855e96214d6694f4ad7c8cfcacaf` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:53.618863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10c050-3db1-4ea1-89aa-8ec324cf1882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d8cf855e96214d6694f4ad7c8cfcacaf)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:53.620323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10c050-3db1-4ea1-89aa-8ec324cf1882 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:53.621991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 587/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu Jun 30 12:21:53 2022] GET /identity/v3/domains/d8cf855e96214d6694f4ad7c8cfcacaf => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:53.629424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:53.713779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:53.737156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab86c3-e843-4e2c-9fb0-c496e51f35d6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:53.772941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d47d4d-086c-4110-a031-356e871642aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:53.773596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d47d4d-086c-4110-a031-356e871642aa None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:53.773873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d47d4d-086c-4110-a031-356e871642aa None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:53.774141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d47d4d-086c-4110-a031-356e871642aa None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:53.774847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d47d4d-086c-4110-a031-356e871642aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:53.776289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d47d4d-086c-4110-a031-356e871642aa None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:53.805457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 586/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:21:53 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:55.872387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dbbf9-b89e-43aa-ac4c-5d872d76cf42 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:55.873026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dbbf9-b89e-43aa-ac4c-5d872d76cf42 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:55.873026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dbbf9-b89e-43aa-ac4c-5d872d76cf42 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:55.874385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 588/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:55.880977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:55.881693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:55.881849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:55.908552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:55.909798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:55.979858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 587/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:56.005728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6dbbf9-b89e-43aa-ac4c-5d872d76cf42 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:56.056665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:56.118455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6dbbf9-b89e-43aa-ac4c-5d872d76cf42 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:56.155313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0acea-8aad-483a-a41a-8fcda4dd08f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:56.156157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0acea-8aad-483a-a41a-8fcda4dd08f8 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:56.156438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0acea-8aad-483a-a41a-8fcda4dd08f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:56.156743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0acea-8aad-483a-a41a-8fcda4dd08f8 None admin] PATH_INFO: `/v3/domains/d8cf855e96214d6694f4ad7c8cfcacaf` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:56.158069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0acea-8aad-483a-a41a-8fcda4dd08f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d8cf855e96214d6694f4ad7c8cfcacaf)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:56.159515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0acea-8aad-483a-a41a-8fcda4dd08f8 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:56.161251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 589/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu Jun 30 12:21:56 2022] GET /identity/v3/domains/d8cf855e96214d6694f4ad7c8cfcacaf => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:56.167881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:56.230789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:56.273991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24dfc4-0510-4fed-aa48-cb8177f9134d None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:56.316591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ccb1f2-f477-4256-9a7b-e47d10769819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:56.317656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccb1f2-f477-4256-9a7b-e47d10769819 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:56.317940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccb1f2-f477-4256-9a7b-e47d10769819 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:56.318214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccb1f2-f477-4256-9a7b-e47d10769819 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:56.318933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ccb1f2-f477-4256-9a7b-e47d10769819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:56.320391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ccb1f2-f477-4256-9a7b-e47d10769819 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:56.344843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 588/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:21:56 2022] POST /identity/v3/users => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:21:58.401598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d50ef1-9fdc-477a-a2e1-25db4e249f07 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:58.402479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d50ef1-9fdc-477a-a2e1-25db4e249f07 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:58.402479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d50ef1-9fdc-477a-a2e1-25db4e249f07 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:58.404694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 590/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:21:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:21:58.413688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:58.414116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:58.414290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:58.434871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:21:58.436048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:21:58.506815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 589/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:21:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:21:58.530757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d50ef1-9fdc-477a-a2e1-25db4e249f07 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:58.654418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d50ef1-9fdc-477a-a2e1-25db4e249f07 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:58.664772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:58.694294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:58.695088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:58.695370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:58.695644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:58.707255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:21:58.714820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:21:58.715946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:21:58.716981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 591/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:21:58.722639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:58.775275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:58.837785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57423144-df48-4862-b5bc-1dd5cec93b0a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:58.879093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:58.879762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:58.880041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:58.880314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:58.881104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:58.881409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:58.882797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:58.891349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 590/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:21:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:58.898053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:59.006683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb370de-222f-4efd-a6a0-6a51b10732df None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:59.046351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:59.047734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:59.048004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:59.048274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] PATH_INFO: `/v3/users/4a5c4ae011f843dc976c6c0ed5e24417` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:59.061801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] RBAC: Authorizing `identity:get_user(user_id=4a5c4ae011f843dc976c6c0ed5e24417)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:59.063379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:59.065055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 592/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:21:58 2022] GET /identity/v3/users/4a5c4ae011f843dc976c6c0ed5e24417 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:59.074870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:59.183278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578173-97b7-459b-923a-42634aa56b53 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:59.192760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:59.221808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:59.222454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:59.222734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:59.223009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:59.297101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:59.298498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:59.398399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 591/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:21:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:21:59.405264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:59.439489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:59.517876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1046de-0858-4b0e-a596-e06c6e45df04 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:59.552828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485ea88-31db-4275-8cc7-8fa4134f48c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:59.553625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485ea88-31db-4275-8cc7-8fa4134f48c9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:59.553905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485ea88-31db-4275-8cc7-8fa4134f48c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:59.554180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485ea88-31db-4275-8cc7-8fa4134f48c9 None admin] PATH_INFO: `/v3/projects/7400dee398db467ca417ce767f2cdd2b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:59.555420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485ea88-31db-4275-8cc7-8fa4134f48c9 None admin] RBAC: Authorizing `identity:get_project(project_id=7400dee398db467ca417ce767f2cdd2b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:59.556905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485ea88-31db-4275-8cc7-8fa4134f48c9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:59.558630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 593/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:21:59 2022] GET /identity/v3/projects/7400dee398db467ca417ce767f2cdd2b => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:21:59.566384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:21:59.578036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:21:59.673460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b5a5e-41af-4f73-8eac-f375021ae2aa None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:21:59.708377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:21:59.709059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:21:59.709335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:21:59.709605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:21:59.710432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7400dee398db467ca417ce767f2cdd2b, user.id=4a5c4ae011f843dc976c6c0ed5e24417) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:21:59.710737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:21:59.712165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29557b38-d861-455a-9daf-ab646e67e7ca None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:21:59.717743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 592/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:21:59 2022] GET /identity/v3/role_assignments?user.id=4a5c4ae011f843dc976c6c0ed5e24417&scope.project.id=7400dee398db467ca417ce767f2cdd2b&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:01.740866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6654c-7b05-45d2-b10f-732c7207e6b1 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:01.741769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6654c-7b05-45d2-b10f-732c7207e6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:01.741769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6654c-7b05-45d2-b10f-732c7207e6b1 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:01.743963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 594/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:22:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:22:01.752872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:01.753605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:01.753727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:01.782026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:22:01.783234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:22:01.855621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 593/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:22:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:22:01.888958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6654c-7b05-45d2-b10f-732c7207e6b1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:01.996067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:02.004782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6654c-7b05-45d2-b10f-732c7207e6b1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:02.041883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:02.042658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:02.042932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:02.043200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:02.054500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:22:02.062024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:22:02.063132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:22:02.063998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 595/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:22:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:22:02.071177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:02.074439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:02.180808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529fca1-b541-4ee2-b39a-7bbd77c0f9a5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:02.219564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:02.220237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:02.220610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:02.220919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:02.221679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:22:02.221985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:02.223370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:02.231623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 594/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:22:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:02.238267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:02.347505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6343d-70da-458b-a0d3-426b2126be6a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:02.383052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:02.383917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:02.384194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:02.384463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] PATH_INFO: `/v3/users/4a5c4ae011f843dc976c6c0ed5e24417` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:02.385816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] RBAC: Authorizing `identity:get_user(user_id=4a5c4ae011f843dc976c6c0ed5e24417)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:02.387260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:02.388959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 596/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:22:02 2022] GET /identity/v3/users/4a5c4ae011f843dc976c6c0ed5e24417 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:02.395437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:02.477091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:02.517955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab712b-a38e-42f3-9c5a-142295a99220 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:02.552670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d8dad-3915-4983-b50b-89629ae90e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:02.553549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d8dad-3915-4983-b50b-89629ae90e74 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:02.553821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d8dad-3915-4983-b50b-89629ae90e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:02.554088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d8dad-3915-4983-b50b-89629ae90e74 None admin] PATH_INFO: `/v3/projects/7400dee398db467ca417ce767f2cdd2b` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:02.555297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d8dad-3915-4983-b50b-89629ae90e74 None admin] RBAC: Authorizing `identity:get_project(project_id=7400dee398db467ca417ce767f2cdd2b)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:02.556766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d8dad-3915-4983-b50b-89629ae90e74 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:02.558454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 595/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:22:02 2022] GET /identity/v3/projects/7400dee398db467ca417ce767f2cdd2b => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:02.565148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:02.595613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:02.674262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6fde0-6cc9-48ae-b830-b33edfbeb466 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:02.710054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c41d15-172c-44a3-9142-0cea2f8dab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:02.710927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c41d15-172c-44a3-9142-0cea2f8dab63 None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:02.711199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c41d15-172c-44a3-9142-0cea2f8dab63 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:02.711462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c41d15-172c-44a3-9142-0cea2f8dab63 None admin] PATH_INFO: `/v3/projects/7400dee398db467ca417ce767f2cdd2b/users/4a5c4ae011f843dc976c6c0ed5e24417/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:02.713703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c41d15-172c-44a3-9142-0cea2f8dab63 None admin] RBAC: Authorizing `identity:create_grant(project_id=7400dee398db467ca417ce767f2cdd2b, user_id=4a5c4ae011f843dc976c6c0ed5e24417, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:02.715120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c41d15-172c-44a3-9142-0cea2f8dab63 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:02.734500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 597/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:22:02 2022] PUT /identity/v3/projects/7400dee398db467ca417ce767f2cdd2b/users/4a5c4ae011f843dc976c6c0ed5e24417/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:22:04.711680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5616e37-bbcf-404a-8433-853f7ca12be2 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:04.712545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5616e37-bbcf-404a-8433-853f7ca12be2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:04.712545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5616e37-bbcf-404a-8433-853f7ca12be2 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:04.714634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 596/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:22:04.722088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:04.722559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:04.722661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:04.740276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:22:04.741500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:22:04.811736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 598/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:22:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:22:04.836247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5616e37-bbcf-404a-8433-853f7ca12be2 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:04.955135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5616e37-bbcf-404a-8433-853f7ca12be2 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:04.991635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:04.998407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:04.999218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:04.999501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:04.999771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:05.013081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:22:05.020830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:22:05.021886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:22:05.022807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 597/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:22:05.028548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:05.044575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:05.131300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613444cf-77c9-49d0-a5a3-6ca6df29d73b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:05.167898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:05.170725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:05.170725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:05.170725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:05.170725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:22:05.170725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:05.171523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:05.180276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 599/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:05.186412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:05.289980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cad2e-2d8a-4ee2-95c7-bbf70cb1173f None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:05.325207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:05.326077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:05.326354 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:05.326628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] PATH_INFO: `/v3/users/4a5c4ae011f843dc976c6c0ed5e24417` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:05.327868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] RBAC: Authorizing `identity:get_user(user_id=4a5c4ae011f843dc976c6c0ed5e24417)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:05.329377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:05.331005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 598/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jun 30 12:22:05 2022] GET /identity/v3/users/4a5c4ae011f843dc976c6c0ed5e24417 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:05.340035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:05.422843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:05.450292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d14f-5c73-41e3-8795-e45068128b40 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:05.493389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:05.494036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:05.494319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:05.494592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:05.566561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:05.568321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:05.645108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:05.671657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 600/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:22:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:22:05.680926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:05.741966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:05.794693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5e4b-805c-40fe-9fdf-3343f4f084e0 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:05.829320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712d0a8-9a6a-481d-b0b2-4cfef4a35ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:05.830117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d0a8-9a6a-481d-b0b2-4cfef4a35ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:05.830391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d0a8-9a6a-481d-b0b2-4cfef4a35ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:05.830658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d0a8-9a6a-481d-b0b2-4cfef4a35ee9 None admin] PATH_INFO: `/v3/projects/7400dee398db467ca417ce767f2cdd2b` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:05.831884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d0a8-9a6a-481d-b0b2-4cfef4a35ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=7400dee398db467ca417ce767f2cdd2b)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:05.833362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d0a8-9a6a-481d-b0b2-4cfef4a35ee9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:05.835058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 599/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jun 30 12:22:05 2022] GET /identity/v3/projects/7400dee398db467ca417ce767f2cdd2b => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:22:05.844306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:22:05.950494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eacef4-774d-4828-9c57-ce2e5c032653 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:22:05.985660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:22:05.986376 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:22:05.986659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:22:05.986932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:22:05.987750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7400dee398db467ca417ce767f2cdd2b, user.id=4a5c4ae011f843dc976c6c0ed5e24417) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:22:05.988061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:22:05.989602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2a31f-4fc4-4ca2-8d0d-dad18d6960be None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:22:05.996642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 601/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:22:05 2022] GET /identity/v3/role_assignments?user.id=4a5c4ae011f843dc976c6c0ed5e24417&scope.project.id=7400dee398db467ca417ce767f2cdd2b&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:22:15.824381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=36 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:22:15.993876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=80 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:02.040663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfd20e-3a33-4866-bf0e-16218652ce7b None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:02.040663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfd20e-3a33-4866-bf0e-16218652ce7b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:02.040663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfd20e-3a33-4866-bf0e-16218652ce7b None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:02.042049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 600/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:02.048414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:02.048990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:02.049138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:02.069692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:02.071238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:02.162265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 602/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:23:02.190065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfd20e-3a33-4866-bf0e-16218652ce7b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:02.351100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfd20e-3a33-4866-bf0e-16218652ce7b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:02.402455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:02.403726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:02.404067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:02.404398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:02.416592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:02.418102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:02.442731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:02.448828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:02.450357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:02.450357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:02.450357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:02.450357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:02.450357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 601/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:02.450970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:02.581289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057aa89-860c-4d25-ae94-f4f77c56adb5 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:02.614812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:02.621951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:02.622632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:02.622917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:02.623237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:02.623934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:02.624238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:02.629906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ca4f8-036e-40b9-b25e-417ba17ee009 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:02.642054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 603/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:02.648381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:02.719674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:02.778583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a50d7-6b18-41dd-8d43-333baa5a2d80 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:02.826615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc726-1460-4c4d-8c01-009f4dd58225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:02.827519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc726-1460-4c4d-8c01-009f4dd58225 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:02.828098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc726-1460-4c4d-8c01-009f4dd58225 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:02.828563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc726-1460-4c4d-8c01-009f4dd58225 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:02.829473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affc726-1460-4c4d-8c01-009f4dd58225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:02.831082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affc726-1460-4c4d-8c01-009f4dd58225 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:02.861168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 602/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jun 30 12:23:02 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:23:05.073798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734cf60-54a3-4606-9998-015651e45473 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:05.073798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734cf60-54a3-4606-9998-015651e45473 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:05.074423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734cf60-54a3-4606-9998-015651e45473 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:05.076680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 604/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:05.081534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:05.082195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:05.082195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:05.100250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:05.101484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:05.199791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 603/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:23:05.223911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734cf60-54a3-4606-9998-015651e45473 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:05.350419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734cf60-54a3-4606-9998-015651e45473 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:05.393461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:05.394363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:05.394637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:05.394973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:05.409054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:23:05.416887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:05.418392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:05.419816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 605/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:05.425092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:05.474756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:05.544176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7faf5-8ad9-4bd1-865c-f5bdacbdfe40 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:05.587907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:05.588893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:05.589357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:05.589739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:05.590773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:05.591119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:05.592661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:05.604568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 604/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:23:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:05.610256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:05.673738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:05.742430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c365e32-5227-4410-ba1d-2b4e438c5944 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:05.784147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:05.785033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:05.785322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:05.785628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:05.798381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:05.799935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:05.825665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:05.826897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:05.829655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:05.829655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:05.829655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:05.829655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:05.829655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 606/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:23:05.833461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:05.945340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:05.949088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbca27f-b925-434b-acb8-530133e65f6a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:05.998697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:05.999509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:05.999886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:06.000167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:06.000975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:06.001287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:06.002779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:06.015971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 605/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:06.021665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:06.173790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2647f7-f826-44dc-a296-cc73de4d62a0 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:06.181398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:06.226431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:06.227698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:06.228136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:06.228651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:06.240132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:06.243099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:23:06.268598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:06.270588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:06.272396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 607/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:23:06.279709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:06.419476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16641b-3f8b-4359-aa0a-973c476f8bce None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:06.469047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:06.469930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:06.470200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:06.470575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:06.471424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:06.471736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:06.473222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:06.491917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 606/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jun 30 12:23:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:06.497753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:06.624135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b6cae-3db3-47c6-830d-12c75464be73 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:06.667210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:06.668031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:06.668315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:06.668588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:06.703781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:06.750478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:06.751592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:06.882138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 608/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:23:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:23:06.889458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:06.910543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:07.009855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eb434-4594-465d-95fc-90532ff84f81 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:07.048999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:07.049850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:07.050089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:07.050449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:07.062097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:07.063920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:07.084514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:07.085591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:07.086500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:07.086500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:07.086500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:07.086500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:07.088492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 607/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:07.094636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:07.161811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:07.209440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa55e2-ea35-4f38-8500-a5a231cfd606 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:07.248876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:07.250400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:07.250813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:07.251158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:07.251964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:07.252297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:07.253896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:07.269081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 609/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:07.274970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:07.361763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:07.399132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f09c-551b-45dd-894a-dc8c07802fad None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:07.442939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:07.443743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:07.444021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:07.444288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:07.451853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:07.453743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:23:07.467285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:07.468424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:07.474526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 608/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:07.475073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:07.593573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95810526-9f24-468b-bbfd-7a9e871ef86b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:07.619795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:07.633751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:07.634534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:07.634813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:07.635180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:07.636057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:07.636442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:07.638341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f7c75-61ed-4733-bd74-d184ec203970 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:07.652294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 610/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:07.659212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:07.770184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203d96e-689d-4e14-a81c-eb95e8657a0e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:07.803273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:07.811994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:07.812814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:07.813228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:07.813568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:07.815011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0783ae884dd1434bbfc76b91457858f8) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:07.815440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:07.817388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646b73-f033-4f32-9963-478fc1ed42c1 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:07.836684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 609/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:23:07 2022] GET /identity/v3/role_assignments?user.id=0783ae884dd1434bbfc76b91457858f8&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:10.008585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9101a2-7f0d-4f14-b5ff-d5c49dae7708 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:10.009211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9101a2-7f0d-4f14-b5ff-d5c49dae7708 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:10.009211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9101a2-7f0d-4f14-b5ff-d5c49dae7708 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:10.010811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 611/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:10.017465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:10.017937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:10.018130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:10.038791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:10.040130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:10.116098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 610/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:10.143263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9101a2-7f0d-4f14-b5ff-d5c49dae7708 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:10.279699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9101a2-7f0d-4f14-b5ff-d5c49dae7708 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:10.318417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:10.319319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:10.319612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:10.319886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:10.331297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:23:10.339475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:10.340978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:10.342139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 612/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:23:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:10.348015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:10.456646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977f02-cf16-4bf9-8ca6-634a117aabf9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:10.480992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:10.494075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:10.494750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:10.495027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:10.495298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:10.496087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] RBAC: Adding query filter params (name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:10.496466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:10.497911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:10.507086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 611/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:23:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:10.513773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:10.561586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:10.622903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7f1ea-3091-4a59-838f-64a45ef8a59d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:10.659632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:10.660470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:10.660814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:10.661158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:10.672846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:10.674418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:10.693486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:10.694657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:10.701629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:10.701629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:10.701629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:10.701629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:10.701629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 613/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:10.701968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:10.807504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50e940-e989-428b-83ec-89fc5384159c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:10.844238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:10.844872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:10.845157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:10.845431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:10.846158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:10.846464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:10.847839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:10.861610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 612/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:10.868409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:10.977498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:10.987042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31d1a6-d9a2-45fb-baf4-85a7c19afda6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:11.020927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:11.021743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:11.022024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:11.022298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:11.029784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:11.031849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:23:11.048100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:11.049480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:11.052040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 614/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:11.058602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:11.112343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:11.172242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee15fc-fddd-4450-ba47-aba3c2f60c5a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:11.207826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:11.208498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:11.208807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:11.209076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:11.209908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:11.210211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:11.211633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:11.231482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 613/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jun 30 12:23:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:11.237462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:11.343947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806c726-9ddb-4357-a753-237e69bbc738 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:11.381375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:11.382135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:11.382431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:11.382697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:11.394890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:11.396715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:11.416801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:11.417764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:11.418749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:11.418749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:11.418749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:11.418749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:11.418749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 615/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:11.424866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:11.535306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65802b-fdba-4890-9d0a-918a8806d9df None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:11.538078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:11.572877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:11.573571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:11.573843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:11.574139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:11.574912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:11.575244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:11.576856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:11.589780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 614/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:11.596118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:11.607128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:11.711332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3531145-b827-4ef7-8af4-bcf83276a235 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:11.747809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:11.748665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:11.748949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:11.749682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:11.757126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:11.758589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:23:11.775110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:11.776422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:11.777395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 616/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:23:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:11.782890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:11.898074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ad11-fb1d-4b37-b183-e967bb2bf504 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:11.936953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:11.937968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:11.938314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:11.938623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:11.939838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:11.940146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:11.941697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f7011-87e8-4e13-aa38-ffa222c7f2ea None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:11.956999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 615/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:23:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:11.963526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:12.045784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:12.085631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd2ab-4246-4300-8e6f-6beaf8206d00 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:12.125033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588a0ef-faa0-44b9-ae71-36cc7719853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:12.126178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588a0ef-faa0-44b9-ae71-36cc7719853a None admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:12.126552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588a0ef-faa0-44b9-ae71-36cc7719853a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:12.126951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588a0ef-faa0-44b9-ae71-36cc7719853a None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0783ae884dd1434bbfc76b91457858f8/roles/359aa45b59614691a2f363547a1e48fa` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:12.144172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588a0ef-faa0-44b9-ae71-36cc7719853a None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=0783ae884dd1434bbfc76b91457858f8, role_id=359aa45b59614691a2f363547a1e48fa)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:12.145809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588a0ef-faa0-44b9-ae71-36cc7719853a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:12.158656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 617/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:23:11 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0783ae884dd1434bbfc76b91457858f8/roles/359aa45b59614691a2f363547a1e48fa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:23:14.198828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cca25-39b3-4e45-bd68-601c684e06c0 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:14.199468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cca25-39b3-4e45-bd68-601c684e06c0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:14.199468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cca25-39b3-4e45-bd68-601c684e06c0 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:14.201015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 616/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:14.207105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:14.207598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:14.207689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:14.228280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:14.229519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:14.304694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 618/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:14.331544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cca25-39b3-4e45-bd68-601c684e06c0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:14.361512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:14.438684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cca25-39b3-4e45-bd68-601c684e06c0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:14.475673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:14.476480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:14.476793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:14.477061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:14.487285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:23:14.494941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:14.495968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:14.496871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 617/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jun 30 12:23:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:14.503626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:14.553826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:14.618153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9151d4-1dc8-4749-aa38-418f6fa2bf1b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:14.651231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:14.651847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:14.652128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:14.652395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:14.653187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] RBAC: Adding query filter params (name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:14.653478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:14.654853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:14.663973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 619/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jun 30 12:23:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:14.669866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:14.760003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:14.776411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4896a-f35c-4c28-8ab0-114d2f968110 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:14.810497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:14.811459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:14.811776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:14.812128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:14.825666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:14.827152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:14.845052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:14.846125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:14.847170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:14.847170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:14.847170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:14.847170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:14.851206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 618/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:14.852437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:14.964188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7df487-6949-45e9-9111-de8eb67d2aad None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:14.975130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:15.004614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:15.005448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:15.005732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:15.006003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:15.006733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:15.007058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:15.008616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:15.022394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 620/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:15.028547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:15.142904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d906982-5a06-46ca-b044-324bd8ad202a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:15.185853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:15.187408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:15.187685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:15.187955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:15.191373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:15.200343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:15.201844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:23:15.216729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:15.217931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:15.219098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 619/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:15.224455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:15.331654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e2330-c75d-4aa5-8b34-c676b4f102a5 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:15.371354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:15.372040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:15.372319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:15.372594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:15.373470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:15.373780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:15.375500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:15.395634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 621/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jun 30 12:23:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:15.401279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:15.505099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b18c0-dfc6-4f74-9bdf-7b8b51d5ce6c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:15.543496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:15.544216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:15.544544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:15.544863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:15.616718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:15.625281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:15.626768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:15.738869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 620/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:23:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:23:15.744951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:15.776015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:15.861964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4249e-4a0c-4b65-96cd-07378824e2f2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:15.897540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:15.898241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:15.898514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:15.898783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:15.909727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:15.911211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:15.930093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:15.931237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:15.932422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:15.932422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:15.932422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:15.932422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:15.932422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 622/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:23:15.938549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:16.052042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:16.058617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d2fa-1396-40fd-a627-798d4ebe60ab None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:16.094731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:16.095444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:16.095729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:16.096024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:16.096814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:16.097164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:16.098653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:16.111469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 621/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:16.117269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:16.208890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:16.232248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c781126-e78b-4d07-993a-d3b13bab1cc6 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:16.269028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:16.269878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:16.270169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:16.270484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:16.278023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:16.279613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:23:16.294248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:16.295208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:16.296221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 623/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:16.301940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:16.414671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbf284-82c7-47fa-b781-af90593a95c6 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:16.450493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:16.451214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:16.451555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:16.451834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:16.452717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:16.453022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:16.454604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10570-4464-4f16-aa26-46bb02577bf9 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:16.468220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 622/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:16.474955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:16.584578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9156f-1bb7-4e34-b77c-4578f9967455 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:16.622815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:16.624037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:16.624037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:16.624822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:16.625479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] RBAC: Adding query filter params (role.id=359aa45b59614691a2f363547a1e48fa, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0783ae884dd1434bbfc76b91457858f8) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:16.625912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:16.627530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba8b97-b0b8-4e2c-8cfb-95c374280425 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:16.648655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 624/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:23:16 2022] GET /identity/v3/role_assignments?user.id=0783ae884dd1434bbfc76b91457858f8&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=359aa45b59614691a2f363547a1e48fa => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:18.854250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231432b-bbd7-4b42-a06e-0d22b60a78fa None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:18.854817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231432b-bbd7-4b42-a06e-0d22b60a78fa None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:18.854817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231432b-bbd7-4b42-a06e-0d22b60a78fa None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:18.856360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 623/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:23:18.862073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:18.862571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:18.862703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:18.868285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:18.883859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:18.885288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:18.963626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 625/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:18.993734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231432b-bbd7-4b42-a06e-0d22b60a78fa None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:19.016062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:19.120463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231432b-bbd7-4b42-a06e-0d22b60a78fa None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:19.161622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:19.162425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:19.162700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:19.162967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:19.175463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:23:19.184388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:19.185569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:19.186642 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 624/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:23:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:19.192795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:19.302717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11564038-9573-4c2b-873e-3c1d72792467 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:19.340429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:19.346067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:19.346952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:19.347432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:19.347722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:19.348527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:19.348857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:19.350330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:19.360157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 626/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:19.366492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:19.454084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:19.482319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789403ed-9ca6-4730-8150-9dce0c7bc32c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:19.520779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:19.521660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:19.521922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:19.522188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:19.534270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:19.535836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:19.555347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:19.556396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:19.558021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:19.558021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:19.558021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:19.558021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:19.558021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 625/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:19.563176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:19.677605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639255-182c-4c64-9b88-4c467decb0e1 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:19.714236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:19.714824 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:19.715140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:19.715503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:19.716315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:19.716620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:19.718906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:19.732327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 627/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:19.739138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:19.852101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e4cb-610c-48ed-9462-28825c4cabce None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:19.881621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:19.891075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:19.891898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:19.892209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:19.892479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:19.901141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:19.902664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:23:19.918540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:19.919902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:19.921105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 626/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:19.926129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:19.999077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:20.054652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895786a3-7a63-4914-a7cd-58630c935234 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:20.095701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:20.096382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:20.096687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:20.096961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:20.097793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:20.098095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:20.099561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:20.118275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 628/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jun 30 12:23:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:20.126382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:20.245949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b53bb5-c2e5-40d6-8fe7-314f73d362c7 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:20.291995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:20.292741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:20.292962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:20.293272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:20.361999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:20.378243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:20.379928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:20.509591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 627/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:23:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:23:20.516966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:20.649512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37d786-7369-4e90-a1f6-c48c05d10985 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:20.665722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:20.691453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:20.692438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:20.692754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:20.693029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:20.707893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:20.709500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:20.729667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:20.730882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:20.736948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:20.736948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:20.736948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:20.736948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:20.736948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 629/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:20.737289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:20.835926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:20.849092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4cfad-8a4d-457b-a1b5-ea4eb6f53e84 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:20.895484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:20.896194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:20.896431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:20.896730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:20.897474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:20.897794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:20.899230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:20.913440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 628/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:20.919819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:21.057074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28f6d-873d-4df0-b26f-8741ca8c7c15 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:21.106007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:21.107307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:21.107708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:21.108282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:21.117900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:21.120028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:23:21.137310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:21.139269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:21.141014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 630/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:21.145702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:21.282181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e421-6cd9-4242-a3a6-8d09341a2c62 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:21.318893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:21.319595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:21.319897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:21.320185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:21.321442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:21.322091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:21.324040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ca8d7-5938-49ae-a51e-33a6724dcdbe None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:21.340819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 629/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:23:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:21.347912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:21.385643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:21.476127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2935b1-d274-4b0f-875b-96476e306ba3 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:21.511777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:21.512441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:21.512756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:21.513046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:21.513897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0783ae884dd1434bbfc76b91457858f8) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:21.514273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:21.516543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fcaee-7b51-47c7-a436-1ff25881f372 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:21.523634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 631/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:23:21 2022] GET /identity/v3/role_assignments?user.id=0783ae884dd1434bbfc76b91457858f8&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:23.530276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af0edd-4dd8-4804-843e-d309c9fd3ce4 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:23.530832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af0edd-4dd8-4804-843e-d309c9fd3ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:23.530832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af0edd-4dd8-4804-843e-d309c9fd3ce4 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:23.532222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 630/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:23.538657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:23.539334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:23.539535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:23.565307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:23.566511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:23.638193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 632/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:23.661053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af0edd-4dd8-4804-843e-d309c9fd3ce4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:23.673711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:23.784243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af0edd-4dd8-4804-843e-d309c9fd3ce4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:23.819557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:23.820333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:23.820605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:23.820903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:23.831006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:23:23.838542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:23.839697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:23.840714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 631/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:23:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:23.847147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:23.867632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:23.964916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad395-5faa-447d-b269-03c93e4f71b9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:24.006003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:24.006680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:24.007073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:24.007404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:24.008229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:24.008610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:24.010170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:24.020171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 633/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:24.026945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:24.100353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:24.143406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdb0cb-bdae-433f-afa1-1d051ab55017 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:24.176994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:24.177797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:24.178092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:24.178355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:24.190545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:24.191998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:24.215179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:24.216173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:24.217141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:24.217141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:24.217141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:24.217141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:24.217141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 632/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:24.223067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:24.305846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:24.329658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475775ef-640d-4378-a9ce-7904e7134a7f None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:24.365600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:24.366199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:24.366479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:24.366757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:24.367480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:24.367791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:24.369215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:24.381085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 634/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:24.387056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:24.510167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9c20-efc0-46af-a403-731bc5cf7798 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:24.531704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:24.546205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:24.547009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:24.547284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:24.547562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:24.554751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:24.556210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:23:24.570310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:24.571335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:24.572378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 633/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:24.577696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:24.686128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda62c-74b1-4dfb-90c2-c42227342362 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:24.724075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:24.724761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:24.725043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:24.725309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:24.726135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:24.726440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:24.727854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:24.736596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:24.746049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 635/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jun 30 12:23:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:24.752471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:24.876575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632301e8-f18c-42f8-8dba-012cd0c470ff None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:24.911778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:24.912560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:24.912868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:24.913132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:24.925167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:24.926637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:24.946204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:24.947224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:24.948294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:24.948294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:24.948294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:24.948294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:24.948796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 634/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:24.953770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:25.068420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0fc35-4510-42e5-8254-051675bfc08d None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:25.105219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:25.105880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:25.106159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:25.106431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:25.107151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:25.107456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:25.109101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:25.120496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 636/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:25.126549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:25.163762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:25.235721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1be9ef-d931-48f1-8ced-426b5fc8c2ca None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:25.272865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:25.273654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:25.273933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:25.274204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:25.281426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:25.282870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:23:25.297391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:25.298440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:25.299794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 635/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:25.304662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:25.365060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:25.411877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284d00d-1c9c-4347-a04a-11cd67d410b2 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:25.453176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:25.453850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:25.454137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:25.454408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:25.455191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:25.455533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:25.458060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47d62b-2bf5-4e22-b2c4-8f58d9c5c3b0 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:25.471651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 637/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:25.477976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:25.580716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:25.591074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9d88e-29aa-4188-8ac3-b15ea8e09f16 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:25.630355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3fe51-25cc-4bcf-849d-6e5c92f70804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:25.631225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3fe51-25cc-4bcf-849d-6e5c92f70804 None admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:25.631502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3fe51-25cc-4bcf-849d-6e5c92f70804 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:25.631768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3fe51-25cc-4bcf-849d-6e5c92f70804 None admin] PATH_INFO: `/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0783ae884dd1434bbfc76b91457858f8/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:25.633985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3fe51-25cc-4bcf-849d-6e5c92f70804 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a65694b54a64a0e8d83a436ed9184cf, user_id=0783ae884dd1434bbfc76b91457858f8, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:25.635426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3fe51-25cc-4bcf-849d-6e5c92f70804 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:25.655083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 636/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jun 30 12:23:25 2022] PUT /identity/v3/projects/7a65694b54a64a0e8d83a436ed9184cf/users/0783ae884dd1434bbfc76b91457858f8/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:23:27.687283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d5434-c2fc-45e1-86ea-7082754a592e None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:27.688184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d5434-c2fc-45e1-86ea-7082754a592e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:27.688184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d5434-c2fc-45e1-86ea-7082754a592e None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:27.690735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 638/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:27.697013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:27.697701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:27.697878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:27.717708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:27.718752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:27.792600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 637/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:27.816981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d5434-c2fc-45e1-86ea-7082754a592e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:27.927522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d5434-c2fc-45e1-86ea-7082754a592e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:27.946893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:27.969310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:27.970246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:27.970584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:27.970885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:27.983988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:23:27.994359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:27.995718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:27.996718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 639/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jun 30 12:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:28.002423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:28.087480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:28.107935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb90d28-2b17-4f77-84a2-52ea17e2be6b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:28.146405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:28.147076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:28.147360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:28.147635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:28.148389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:28.148732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:28.150110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:28.158585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 638/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jun 30 12:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:28.166846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:28.270249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa64f0-0ceb-42a0-b027-ee8668a62e51 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:28.309917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:28.310705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:28.311244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:28.311825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:28.322745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:28.324190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:28.341615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:28.342966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:28.343949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:28.343949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:28.343949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:28.343949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:28.343949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 640/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:28.352204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:28.456688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78883476-08c7-4281-817d-22fd7d73043e None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:28.491462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:28.492115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:28.492392 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:28.492666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:28.493417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:28.493720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:28.495087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:28.507028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 639/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:28.513752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:28.553098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:28.629252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a53f6-ea3e-4997-84ac-6336a95c9a23 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:28.665499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:28.666321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:28.666599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:28.666867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:28.674430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:28.675883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:23:28.690685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:28.692567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:28.694394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 641/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:28.701282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:28.707502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:28.809447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546d763-92f0-48f1-a072-f684e450f83c None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:28.845353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:28.846152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:28.846433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:28.846707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:28.847551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:28.847863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:28.849336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:28.867122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 640/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jun 30 12:23:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:28.873913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:28.981173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:28.993579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ee256-9253-4059-8f59-958d7b3384cc None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:29.034177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:29.034820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:29.035095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:29.035355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:29.109543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:29.110917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:29.209394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:29.226530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 642/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jun 30 12:23:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:23:29.232841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:29.309479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:29.350581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddba948-8ce8-46b0-8ca1-5812a3bb0503 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:29.385284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:29.386056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:29.386293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:29.386557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:29.397105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:29.398568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:23:29.416065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:29.417106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:29.418174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:23:29.418174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:23:29.418174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:23:29.418174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:29.420824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 641/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:29.423292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:29.550431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ce4e9-6ec7-4470-8078-ef0c29b4aea9 None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:29.593330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:29.593995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:29.594277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:29.594550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:29.595274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:29.595581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:29.596993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:29.609783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 643/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jun 30 12:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:29.616413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:29.724126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2707c3-b9bd-488c-8bf3-9ceeb4536c6b None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:29.740158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:29.762114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:29.762899 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:29.763180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:29.763452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:29.770329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:29.771778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:23:29.785783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:29.786728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: ERROR keystone.server.flask.application Jun 30 12:23:29.787640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 642/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jun 30 12:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:29.793060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:29.855039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:29.916979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fad3-3e73-44ae-a972-4fa846a6010b None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:29.954733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:29.955385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:29.955660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:29.955929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:29.956900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:29.957173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:29.958641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194482e-2213-463e-9675-cb00d41c8483 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:29.973312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 644/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jun 30 12:23:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:29.980893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:30.099610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712ce76-cf30-4e4b-90b2-2f945d68e058 None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:30.134376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:30.135027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:30.135308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:30.135581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:30.136411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] RBAC: Adding query filter params (role.id=bb98be56cb17479cb7d590fc8df81574, scope.project.id=7a65694b54a64a0e8d83a436ed9184cf, user.id=0783ae884dd1434bbfc76b91457858f8) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:30.136750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:30.138193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20441f-a17f-48ba-8245-9ec462d22e2f None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:30.144128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 643/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jun 30 12:23:29 2022] GET /identity/v3/role_assignments?user.id=0783ae884dd1434bbfc76b91457858f8&scope.project.id=7a65694b54a64a0e8d83a436ed9184cf&role.id=bb98be56cb17479cb7d590fc8df81574 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:32.179664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274f0e-9a66-459d-96d8-0fa512617fa4 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:32.180117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274f0e-9a66-459d-96d8-0fa512617fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:32.180219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274f0e-9a66-459d-96d8-0fa512617fa4 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:32.181649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 645/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:32.187105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:32.187561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:32.187655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:32.208846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:32.210619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:32.248542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:32.287938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 644/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:32.320366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14274f0e-9a66-459d-96d8-0fa512617fa4 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:32.431889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14274f0e-9a66-459d-96d8-0fa512617fa4 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:32.443372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:32.469545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:32.470353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:32.470630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:32.470903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:32.471543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:32.472940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:32.485334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:32.486461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:32.487702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 646/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:23:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:32.492790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:32.599706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0e125-bb4f-4806-ae4e-ccb07641af91 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:32.635296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:32.635952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:32.636227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:32.636501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:32.637256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:32.637565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:32.639012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0dc60-e30f-42ff-8c5d-2576a3b39c3b None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:32.644071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 645/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jun 30 12:23:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:32.650622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:32.768714 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc02b91-a12a-464c-b832-c7f1bb1bc50e None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:32.803330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:32.803956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:32.804265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:32.804537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] PATH_INFO: `/v3/services` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:32.805274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:32.805576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:32.806951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9760b-7453-4096-97c4-4d428fa7b571 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:32.811595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 647/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jun 30 12:23:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:34.828839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c7734-de02-4574-b38d-d7e04461d697 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:34.829698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c7734-de02-4574-b38d-d7e04461d697 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:34.829698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c7734-de02-4574-b38d-d7e04461d697 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:34.831907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 646/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:34.837277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77df46-76d7-4ead-a757-e52ad5f03352 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:34.837732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77df46-76d7-4ead-a757-e52ad5f03352 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:34.837833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77df46-76d7-4ead-a757-e52ad5f03352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:34.856274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-dc77df46-76d7-4ead-a757-e52ad5f03352 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:34.857554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dc77df46-76d7-4ead-a757-e52ad5f03352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:34.890370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:34.933155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 648/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:34.957255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c7734-de02-4574-b38d-d7e04461d697 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:34.998320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:35.084091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c7734-de02-4574-b38d-d7e04461d697 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:35.120005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a98f56-8ea3-4c83-b9a6-65791ad48722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:35.120676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a98f56-8ea3-4c83-b9a6-65791ad48722 None admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:35.120971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a98f56-8ea3-4c83-b9a6-65791ad48722 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:35.121250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a98f56-8ea3-4c83-b9a6-65791ad48722 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:35.121966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a98f56-8ea3-4c83-b9a6-65791ad48722 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:35.123409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a98f56-8ea3-4c83-b9a6-65791ad48722 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:35.140584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 647/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jun 30 12:23:34 2022] POST /identity/v3/services => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:23:37.169895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e4e3-58e7-450f-b2df-fc106b519e2b None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:37.170470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e4e3-58e7-450f-b2df-fc106b519e2b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:37.170470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e4e3-58e7-450f-b2df-fc106b519e2b None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:37.171971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 649/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:37.179131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:37.179567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:37.179666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:37.199057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:37.200230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:37.277896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 648/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:23:37.305773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e4e3-58e7-450f-b2df-fc106b519e2b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:37.423987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e4e3-58e7-450f-b2df-fc106b519e2b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:37.471069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:37.471873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:37.472166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:37.472430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:37.473121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:37.474509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:37.486829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:37.487983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:37.489004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 650/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:23:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:37.494380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:37.513502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:37.594067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccea7d-4d51-4146-9038-722896eb6dd6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:37.630260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:37.630916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:37.631199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:37.631466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:37.632171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:37.632473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:37.633884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:37.639324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 649/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jun 30 12:23:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:37.645615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:37.686957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:37.761702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ffe6f-9e7c-4ced-8fa9-a6c1e838887c None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:37.797040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:37.797703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:37.797985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:37.798259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:37.799056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] RBAC: Adding query filter params (interface=public, service_id=67b659ef5704417c92eaa01d28a1dc27, region_id=RegionOne) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:37.799366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:37.800772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85841f1-55a6-442c-a6b5-1998f33021f4 None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:37.805974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 651/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jun 30 12:23:37 2022] GET /identity/v3/endpoints?service_id=67b659ef5704417c92eaa01d28a1dc27&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:23:37.811983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:37.896096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:37.915492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcde1ce-1a3e-40ee-bc1c-4377f37c1a0a None admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:37.951441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10b21-8a2c-4a4f-8e4c-c2381110c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:37.952112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10b21-8a2c-4a4f-8e4c-c2381110c052 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:37.952396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10b21-8a2c-4a4f-8e4c-c2381110c052 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:37.952669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10b21-8a2c-4a4f-8e4c-c2381110c052 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:37.953400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f10b21-8a2c-4a4f-8e4c-c2381110c052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:37.954784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f10b21-8a2c-4a4f-8e4c-c2381110c052 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:37.962817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 650/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jun 30 12:23:37 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:23:39.986661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d0d68-1aca-4769-844e-356b552c119a None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:39.987225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d0d68-1aca-4769-844e-356b552c119a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:39.987225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d0d68-1aca-4769-844e-356b552c119a None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:39.988613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 652/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:23:39.994958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:39.995674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:39.995819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:40.016317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:23:40.017562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:23:40.092142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 651/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:23:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:23:40.121307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d0d68-1aca-4769-844e-356b552c119a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:40.230137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:40.238609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d0d68-1aca-4769-844e-356b552c119a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:40.274012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:40.274813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:40.275093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:40.275367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:40.276014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:40.277417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:23:40.289672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:23:40.290814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application with Lock( Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:23:40.291959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: ERROR keystone.server.flask.application Jun 30 12:23:40.296115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 653/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jun 30 12:23:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:23:40.297338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:40.374200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:40.400548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d95ce8-93b1-4e4a-8c98-21760b711b56 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:40.437311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:40.438332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:40.438609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:40.438878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] PATH_INFO: `/v3/services` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:40.439588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:23:40.439892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:40.441307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72606a-5704-4dd4-87d1-97b7b79e2034 None admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:40.446739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 652/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jun 30 12:23:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:23:40.452918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:23:40.566763 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a27bfe-f39b-4b3d-b021-b04a1fbe2e4a None admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:23:40.606254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80224b-669b-4cf0-8e0e-464b939c706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:23:40.606902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80224b-669b-4cf0-8e0e-464b939c706d None admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:23:40.607164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80224b-669b-4cf0-8e0e-464b939c706d None admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:23:40.607473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80224b-669b-4cf0-8e0e-464b939c706d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:23:40.608313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80224b-669b-4cf0-8e0e-464b939c706d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:23:40.609720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80224b-669b-4cf0-8e0e-464b939c706d None admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:23:40.632776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 654/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jun 30 12:23:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:23:50.443626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=30 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:23:50.628444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:44.145015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cc2c0-1596-4e8f-9c17-476d8c812fad None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:44.145603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cc2c0-1596-4e8f-9c17-476d8c812fad None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:44.145603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cc2c0-1596-4e8f-9c17-476d8c812fad None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:44.152931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125ad2-e3c0-410e-b096-c2ea770df01d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:44.153493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125ad2-e3c0-410e-b096-c2ea770df01d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:44.153493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125ad2-e3c0-410e-b096-c2ea770df01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:44.157046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 653/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:24:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:24:44.245399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-22125ad2-e3c0-410e-b096-c2ea770df01d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:44.246683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-22125ad2-e3c0-410e-b096-c2ea770df01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:44.351693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 655/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:24:44.402071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707ff1f-acb3-4633-b7b3-1bef642172d5 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:44.402854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707ff1f-acb3-4633-b7b3-1bef642172d5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:44.403373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707ff1f-acb3-4633-b7b3-1bef642172d5 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:44.405473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 654/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu Jun 30 12:24:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:24:44.411839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e0e9-baa3-4eab-adf2-791b767c3ef9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:44.412126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e0e9-baa3-4eab-adf2-791b767c3ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:44.412403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e0e9-baa3-4eab-adf2-791b767c3ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:44.505526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-16d9e0e9-baa3-4eab-adf2-791b767c3ef9 None None] MFA Rules not processed for user `94b91e199622457787fa2a7e78d743c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:44.507433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-16d9e0e9-baa3-4eab-adf2-791b767c3ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:44.552805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:44.651169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 656/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu Jun 30 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:24:44.669779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707ff1f-acb3-4633-b7b3-1bef642172d5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:24:44.672106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2707ff1f-acb3-4633-b7b3-1bef642172d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:44.825471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707ff1f-acb3-4633-b7b3-1bef642172d5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:24:44.906307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:24:44.907294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:44.907800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:44.908290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:44.910142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:44.974122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:45.036574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:24:45.038486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077fb55-3dc3-45b1-b413-57785925057f service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:24:45.060745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 655/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:24:47.415917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56373f47-20a4-41ef-b66e-4b853d80a214 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:47.416492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56373f47-20a4-41ef-b66e-4b853d80a214 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:47.416492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56373f47-20a4-41ef-b66e-4b853d80a214 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:47.418218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 657/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:24:47.423458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9d876-5afb-4b09-82fb-653eeb1426f0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:47.423896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9d876-5afb-4b09-82fb-653eeb1426f0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:47.423896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9d876-5afb-4b09-82fb-653eeb1426f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:47.513914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-fad9d876-5afb-4b09-82fb-653eeb1426f0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:47.515162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fad9d876-5afb-4b09-82fb-653eeb1426f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:47.626478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 656/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:24:47.664073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56373f47-20a4-41ef-b66e-4b853d80a214 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:24:47.665709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-56373f47-20a4-41ef-b66e-4b853d80a214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:47.770529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56373f47-20a4-41ef-b66e-4b853d80a214 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:24:47.847239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:47.850322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:24:47.851541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:47.851965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:47.852287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:47.854184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:47.969060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:24:47.970735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41b63-28cd-4f8a-bf69-759b58f4a29e service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:24:47.989795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 658/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:24:50.367475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2525d0-f46a-489d-8f8b-7416484507bd None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:50.368031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2525d0-f46a-489d-8f8b-7416484507bd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:50.368031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2525d0-f46a-489d-8f8b-7416484507bd None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:50.369544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 657/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:24:50.376433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebc7e9-be86-46bd-a957-caca9436c0d6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:50.377083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebc7e9-be86-46bd-a957-caca9436c0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:50.377083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebc7e9-be86-46bd-a957-caca9436c0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:50.445062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-34ebc7e9-be86-46bd-a957-caca9436c0d6 None None] MFA Rules not processed for user `d703bbb25adb40b5a2c4ac09ee36e0db`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:50.446275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-34ebc7e9-be86-46bd-a957-caca9436c0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:50.499303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:50.544809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 659/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:24:50.608568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abf028-0a54-495a-8173-5fadc4789085 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:50.608908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abf028-0a54-495a-8173-5fadc4789085 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:50.609193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abf028-0a54-495a-8173-5fadc4789085 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:50.610796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 658/1317] 10.222.0.57 () {54 vars in 964 bytes} [Thu Jun 30 12:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:24:50.616806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beafb5-b4e1-4012-842e-593572c831b3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:50.617102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beafb5-b4e1-4012-842e-593572c831b3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:50.617377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beafb5-b4e1-4012-842e-593572c831b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:50.703625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-63beafb5-b4e1-4012-842e-593572c831b3 None None] MFA Rules not processed for user `94b91e199622457787fa2a7e78d743c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:50.705612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-63beafb5-b4e1-4012-842e-593572c831b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:50.789233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 660/1318] 10.222.0.57 () {58 vars in 1061 bytes} [Thu Jun 30 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:24:50.798447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abf028-0a54-495a-8173-5fadc4789085 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:24:50.799988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-25abf028-0a54-495a-8173-5fadc4789085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:50.810321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:50.910588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abf028-0a54-495a-8173-5fadc4789085 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:24:50.992902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:24:50.993874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:50.994155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:50.994427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:50.996148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:51.063490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:51.113902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:24:51.115364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2f4c5-eda9-41d3-9124-2fdfa09abd2a service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:24:51.133568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 659/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:24:54.088678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871195c6-71c7-4d19-965d-38a00acfb65e None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:54.089235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871195c6-71c7-4d19-965d-38a00acfb65e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:54.089235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871195c6-71c7-4d19-965d-38a00acfb65e None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:54.090694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 661/1320] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:24:54.097136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fb896-9dec-4aee-9f66-aff682fb2354 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:54.097604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fb896-9dec-4aee-9f66-aff682fb2354 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:54.097710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fb896-9dec-4aee-9f66-aff682fb2354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:54.168566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-549fb896-9dec-4aee-9f66-aff682fb2354 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:54.169767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-549fb896-9dec-4aee-9f66-aff682fb2354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:54.279028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 660/1321] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:24:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:24:54.322304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871195c6-71c7-4d19-965d-38a00acfb65e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:24:54.324149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-871195c6-71c7-4d19-965d-38a00acfb65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:54.375868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:54.434825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871195c6-71c7-4d19-965d-38a00acfb65e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:24:54.509588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:24:54.510232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:54.510621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:54.510899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:54.512537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:54.640144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:54.654978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:24:54.656428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3f0de-362a-4423-b06b-5288d918f5bf service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:24:54.676905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 662/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:24:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:24:57.426032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0beb3e-836e-462b-93b9-1300a60ba1d3 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:57.426623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0beb3e-836e-462b-93b9-1300a60ba1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:57.426623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0beb3e-836e-462b-93b9-1300a60ba1d3 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:57.427854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 661/1323] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:24:57.433280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31a9c-87d9-498f-8e50-d1dd48b96b5c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:57.433743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31a9c-87d9-498f-8e50-d1dd48b96b5c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:57.433846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31a9c-87d9-498f-8e50-d1dd48b96b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:57.500813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-afa31a9c-87d9-498f-8e50-d1dd48b96b5c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:24:57.502021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-afa31a9c-87d9-498f-8e50-d1dd48b96b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:57.606325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 663/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:24:57.642857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0beb3e-836e-462b-93b9-1300a60ba1d3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:24:57.644517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7d0beb3e-836e-462b-93b9-1300a60ba1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:57.665699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:57.759918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0beb3e-836e-462b-93b9-1300a60ba1d3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:24:57.843614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:24:57.844279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:24:57.844711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:24:57.844930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:24:57.846597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:24:57.920811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:24:57.963952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:24:57.965521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55e7a-2d2e-4023-a55e-5777d49b20d8 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:24:57.996098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 662/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:00.067389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a2851-fc97-4f66-892f-f3d09f3f81f1 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:00.067921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a2851-fc97-4f66-892f-f3d09f3f81f1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:00.067921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a2851-fc97-4f66-892f-f3d09f3f81f1 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:00.069500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 664/1326] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:00.075134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a069f-ee70-4522-b9ab-68274766237c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:00.075616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a069f-ee70-4522-b9ab-68274766237c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:00.075718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a069f-ee70-4522-b9ab-68274766237c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:00.142359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-1f0a069f-ee70-4522-b9ab-68274766237c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:00.143489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1f0a069f-ee70-4522-b9ab-68274766237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:00.245922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 663/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:00.282498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a2851-fc97-4f66-892f-f3d09f3f81f1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:00.284074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-bb4a2851-fc97-4f66-892f-f3d09f3f81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:00.349832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:00.395158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a2851-fc97-4f66-892f-f3d09f3f81f1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:00.472695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:00.473402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:00.473684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:00.473960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:00.475724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:00.594383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:00.597080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:00.598978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39944eed-3662-4cc1-9e14-8f457a6ed8e4 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:00.618039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 665/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:02.708993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7fde6-191d-43d3-8ae2-b56b6b4a37bd None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:02.709591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7fde6-191d-43d3-8ae2-b56b6b4a37bd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:02.709591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7fde6-191d-43d3-8ae2-b56b6b4a37bd None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:02.710919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 664/1329] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:02.716483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cc1f5-eefb-4249-bbfb-7a4323f2a0c7 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:02.716940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cc1f5-eefb-4249-bbfb-7a4323f2a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:02.717056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cc1f5-eefb-4249-bbfb-7a4323f2a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:02.783702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f26cc1f5-eefb-4249-bbfb-7a4323f2a0c7 None None] MFA Rules not processed for user `d703bbb25adb40b5a2c4ac09ee36e0db`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:02.784994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f26cc1f5-eefb-4249-bbfb-7a4323f2a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:02.890891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 666/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:02.928371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7fde6-191d-43d3-8ae2-b56b6b4a37bd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:02.929996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6dd7fde6-191d-43d3-8ae2-b56b6b4a37bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:02.965927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:03.045841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7fde6-191d-43d3-8ae2-b56b6b4a37bd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:03.121802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:03.122380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:03.122658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:03.122930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:03.124607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:03.210531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:03.245933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:03.247416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9132d19-2226-4fb0-8bb4-e31dccac4d38 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:03.266235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 665/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:05.552251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bd1e5-952d-4ba6-ba39-6712d518adda None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:05.552893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bd1e5-952d-4ba6-ba39-6712d518adda None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:05.552893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bd1e5-952d-4ba6-ba39-6712d518adda None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:05.554243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 667/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:05.560116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e5cd0-a89c-4ed3-8f8e-58e116474c45 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:05.560570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e5cd0-a89c-4ed3-8f8e-58e116474c45 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:05.560708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e5cd0-a89c-4ed3-8f8e-58e116474c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:05.635980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2a6e5cd0-a89c-4ed3-8f8e-58e116474c45 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:05.637298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2a6e5cd0-a89c-4ed3-8f8e-58e116474c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:05.737774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 666/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:05.777184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bd1e5-952d-4ba6-ba39-6712d518adda None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:05.779617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-461bd1e5-952d-4ba6-ba39-6712d518adda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:05.863932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:05.902690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bd1e5-952d-4ba6-ba39-6712d518adda None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:05.992302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:05.992995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:05.993271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:05.993541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:05.995226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:06.105742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:06.107189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40133b2c-e670-4eb7-8b7c-f2e511d4ffce service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:06.117178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:06.125532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 668/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:08.204795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc50916-e978-4a1b-9ca1-e1655bee6fc0 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:08.205277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc50916-e978-4a1b-9ca1-e1655bee6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:08.205369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc50916-e978-4a1b-9ca1-e1655bee6fc0 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:08.207437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 667/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:08.213820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ded70b-cbd6-4eb2-9818-b79ccf00f9c4 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:08.214245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ded70b-cbd6-4eb2-9818-b79ccf00f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:08.214407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ded70b-cbd6-4eb2-9818-b79ccf00f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:08.281829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-70ded70b-cbd6-4eb2-9818-b79ccf00f9c4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:08.283033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-70ded70b-cbd6-4eb2-9818-b79ccf00f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:08.382995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 669/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:08.426882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc50916-e978-4a1b-9ca1-e1655bee6fc0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:08.428337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ccc50916-e978-4a1b-9ca1-e1655bee6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:08.471293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:08.539101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc50916-e978-4a1b-9ca1-e1655bee6fc0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:08.615043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:08.615697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:08.615977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:08.616256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:08.617935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:08.709533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:08.738566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:08.740029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d4cd-88b6-4bb7-9e4e-ac1cd3065f6b service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:08.759478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 668/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:11.244683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc20d86-b72b-4675-9508-36fdd5c68165 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:11.245596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc20d86-b72b-4675-9508-36fdd5c68165 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:11.245596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc20d86-b72b-4675-9508-36fdd5c68165 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:11.247888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 670/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:11.254370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b4577-0d01-4233-86f2-d09978c3e828 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:11.255086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b4577-0d01-4233-86f2-d09978c3e828 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:11.255285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b4577-0d01-4233-86f2-d09978c3e828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:11.333597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7e8b4577-0d01-4233-86f2-d09978c3e828 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:11.334821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7e8b4577-0d01-4233-86f2-d09978c3e828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:11.438432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 669/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:11.474952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc20d86-b72b-4675-9508-36fdd5c68165 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:11.477389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2fc20d86-b72b-4675-9508-36fdd5c68165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:11.565284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:11.596091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc20d86-b72b-4675-9508-36fdd5c68165 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:11.672060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:11.672739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:11.673017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:11.673289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:11.674933 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:11.790644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:11.792011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b0740-ee99-4576-b1af-55ea6ad32087 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:11.811175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 671/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:15.976983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b22a-e91c-413c-8e60-1ad2ab216b06 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:15.977546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b22a-e91c-413c-8e60-1ad2ab216b06 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:15.977546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b22a-e91c-413c-8e60-1ad2ab216b06 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:15.979113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 670/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:15.984402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe91db-3929-46c9-b63c-eec742802570 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:15.984868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe91db-3929-46c9-b63c-eec742802570 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:15.985010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe91db-3929-46c9-b63c-eec742802570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:15.991362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:16.062481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-fbbe91db-3929-46c9-b63c-eec742802570 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:16.063700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fbbe91db-3929-46c9-b63c-eec742802570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:16.179232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 672/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:16.218668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b22a-e91c-413c-8e60-1ad2ab216b06 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:16.220926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bb11b22a-e91c-413c-8e60-1ad2ab216b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:16.271749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:16.320703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b22a-e91c-413c-8e60-1ad2ab216b06 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:16.325281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a4b22-3ba5-4283-8aba-b238f6fdb945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:16.325974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4b22-3ba5-4283-8aba-b238f6fdb945 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:16.326281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4b22-3ba5-4283-8aba-b238f6fdb945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:16.326554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4b22-3ba5-4283-8aba-b238f6fdb945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:16.335963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a4b22-3ba5-4283-8aba-b238f6fdb945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:16.337415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a4b22-3ba5-4283-8aba-b238f6fdb945 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:16.347700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 671/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:25:19.053914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93f640-fb96-4b61-a22d-c589b2f56271 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:19.054514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93f640-fb96-4b61-a22d-c589b2f56271 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:19.054514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93f640-fb96-4b61-a22d-c589b2f56271 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:19.055995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 673/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:19.062257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e3de-2136-4baf-934d-888240eb34a5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:19.062712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e3de-2136-4baf-934d-888240eb34a5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:19.062829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e3de-2136-4baf-934d-888240eb34a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:19.095728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d421e3de-2136-4baf-934d-888240eb34a5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:19.097054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d421e3de-2136-4baf-934d-888240eb34a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:19.186465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 672/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:19.224868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93f640-fb96-4b61-a22d-c589b2f56271 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:19.239188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93f640-fb96-4b61-a22d-c589b2f56271 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:19.243162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc1a9b-8bdd-483e-aba2-b52dbb908794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:19.243967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc1a9b-8bdd-483e-aba2-b52dbb908794 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:19.244121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc1a9b-8bdd-483e-aba2-b52dbb908794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:19.244471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc1a9b-8bdd-483e-aba2-b52dbb908794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:19.255126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc1a9b-8bdd-483e-aba2-b52dbb908794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:19.256580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc1a9b-8bdd-483e-aba2-b52dbb908794 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:19.267316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 674/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:23.538955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ef03-523b-47ac-9f3f-82f92fad5e55 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:23.539545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ef03-523b-47ac-9f3f-82f92fad5e55 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:23.539545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ef03-523b-47ac-9f3f-82f92fad5e55 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:23.540963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 673/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:25:23.546556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247659d-6131-4070-834b-c3c345adaaa5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:23.547177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247659d-6131-4070-834b-c3c345adaaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:23.547177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247659d-6131-4070-834b-c3c345adaaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:23.567046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3247659d-6131-4070-834b-c3c345adaaa5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:23.568270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3247659d-6131-4070-834b-c3c345adaaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:23.577794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 675/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:23.621126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878ef03-523b-47ac-9f3f-82f92fad5e55 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:23.632963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878ef03-523b-47ac-9f3f-82f92fad5e55 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:23.637225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244afce9-976f-4aa0-bdda-b16bbe18a04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:23.637978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244afce9-976f-4aa0-bdda-b16bbe18a04b service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:23.638265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244afce9-976f-4aa0-bdda-b16bbe18a04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:23.638540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244afce9-976f-4aa0-bdda-b16bbe18a04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:23.655013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244afce9-976f-4aa0-bdda-b16bbe18a04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:23.657539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244afce9-976f-4aa0-bdda-b16bbe18a04b service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:23.670862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 674/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:28.723034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31b05c-b941-4673-8fbf-dcff3ea3a19e None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:28.724265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31b05c-b941-4673-8fbf-dcff3ea3a19e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:28.724780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31b05c-b941-4673-8fbf-dcff3ea3a19e None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:28.726830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 676/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:25:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:28.733417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2d91-9fb3-4c1d-9983-f74f8f36d941 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:28.734500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2d91-9fb3-4c1d-9983-f74f8f36d941 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:28.734984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2d91-9fb3-4c1d-9983-f74f8f36d941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:28.816090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-75af2d91-9fb3-4c1d-9983-f74f8f36d941 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:28.817959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-75af2d91-9fb3-4c1d-9983-f74f8f36d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:28.847096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:28.915386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 675/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:32.903693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b476d-35a8-46da-8de0-1e0baf238752 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:32.904274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b476d-35a8-46da-8de0-1e0baf238752 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:32.904274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b476d-35a8-46da-8de0-1e0baf238752 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:32.905900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 677/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:32.911046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283edb5-a116-4ae0-a19e-e35a858d1b04 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:32.911496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283edb5-a116-4ae0-a19e-e35a858d1b04 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:32.911599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283edb5-a116-4ae0-a19e-e35a858d1b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:32.931629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c283edb5-a116-4ae0-a19e-e35a858d1b04 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:32.935602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c283edb5-a116-4ae0-a19e-e35a858d1b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:32.943992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 676/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:33.561222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=72 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:34.496643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed7db-e2ce-4bd9-bb5d-786820c4cfbc None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:34.497310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed7db-e2ce-4bd9-bb5d-786820c4cfbc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:34.497310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed7db-e2ce-4bd9-bb5d-786820c4cfbc None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:34.498628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 678/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:34.504805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee4eac-e3be-4ac7-82bd-74714f74986a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:34.505240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee4eac-e3be-4ac7-82bd-74714f74986a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:34.505391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee4eac-e3be-4ac7-82bd-74714f74986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:34.527920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c6ee4eac-e3be-4ac7-82bd-74714f74986a None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:34.529175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c6ee4eac-e3be-4ac7-82bd-74714f74986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:34.538871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 677/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:38.200889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17b5e4-f4a2-427e-9a97-ec58b6dece7c None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:38.201767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17b5e4-f4a2-427e-9a97-ec58b6dece7c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:38.202226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17b5e4-f4a2-427e-9a97-ec58b6dece7c None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:38.208686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 679/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:38.210212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3828-122c-409e-9618-42afab5b33c1 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:38.210641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3828-122c-409e-9618-42afab5b33c1 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:38.210919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3828-122c-409e-9618-42afab5b33c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:38.230820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-604f3828-122c-409e-9618-42afab5b33c1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:38.232427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-604f3828-122c-409e-9618-42afab5b33c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:38.241097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 678/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:38.297881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b4902-b429-499e-ad3e-5498aafd28c7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:38.298510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b4902-b429-499e-ad3e-5498aafd28c7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:38.298935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b4902-b429-499e-ad3e-5498aafd28c7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:38.301070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 680/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu Jun 30 12:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:25:38.309835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da6ec1-0fef-4ddc-a8e7-250935d7e48f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:38.310336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da6ec1-0fef-4ddc-a8e7-250935d7e48f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:38.310795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da6ec1-0fef-4ddc-a8e7-250935d7e48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:38.368119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-33da6ec1-0fef-4ddc-a8e7-250935d7e48f None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:38.369421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-33da6ec1-0fef-4ddc-a8e7-250935d7e48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:38.454840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 679/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Jun 30 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:25:38.463472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b4902-b429-499e-ad3e-5498aafd28c7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:38.478695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b4902-b429-499e-ad3e-5498aafd28c7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:38.485364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5e1b2-1fd2-4b9b-b7a2-5d9e062853f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:38.486119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5e1b2-1fd2-4b9b-b7a2-5d9e062853f1 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:38.486476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5e1b2-1fd2-4b9b-b7a2-5d9e062853f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:38.486823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5e1b2-1fd2-4b9b-b7a2-5d9e062853f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:38.499560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5e1b2-1fd2-4b9b-b7a2-5d9e062853f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:38.502557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5e1b2-1fd2-4b9b-b7a2-5d9e062853f1 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:38.526910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 681/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:25:40.748941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d058a-dd84-4cb5-8e15-d432276039a7 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:40.749501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d058a-dd84-4cb5-8e15-d432276039a7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:40.749501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d058a-dd84-4cb5-8e15-d432276039a7 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:40.751031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 680/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:40.756391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b372098-f5a6-4b7f-8c51-9830deb10c43 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:40.756856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b372098-f5a6-4b7f-8c51-9830deb10c43 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:40.757004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b372098-f5a6-4b7f-8c51-9830deb10c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:40.776898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8b372098-f5a6-4b7f-8c51-9830deb10c43 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:40.778202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8b372098-f5a6-4b7f-8c51-9830deb10c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:40.788681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 682/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:40.831943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d058a-dd84-4cb5-8e15-d432276039a7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:40.846703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d058a-dd84-4cb5-8e15-d432276039a7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:40.852175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3cc06-6a02-49ee-8b75-c77c32d02998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:40.852851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3cc06-6a02-49ee-8b75-c77c32d02998 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:40.853141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3cc06-6a02-49ee-8b75-c77c32d02998 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:40.853562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3cc06-6a02-49ee-8b75-c77c32d02998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:40.861980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3cc06-6a02-49ee-8b75-c77c32d02998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:40.863599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3cc06-6a02-49ee-8b75-c77c32d02998 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:40.873708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 681/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:43.222258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1da87-a3c3-4ea9-8eb9-49ebf34eeb54 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:43.222706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1da87-a3c3-4ea9-8eb9-49ebf34eeb54 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:43.222803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1da87-a3c3-4ea9-8eb9-49ebf34eeb54 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:43.224311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 683/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:43.230053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be567e-2d02-4c70-880e-35d982fbbf20 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:43.230465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be567e-2d02-4c70-880e-35d982fbbf20 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:43.230603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be567e-2d02-4c70-880e-35d982fbbf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:43.249645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d9be567e-2d02-4c70-880e-35d982fbbf20 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:43.250866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d9be567e-2d02-4c70-880e-35d982fbbf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:43.259803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 682/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:43.309225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f6e9f-784f-4bad-b014-693250b263f0 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:43.309513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f6e9f-784f-4bad-b014-693250b263f0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:43.309787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f6e9f-784f-4bad-b014-693250b263f0 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:43.311163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 684/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu Jun 30 12:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:43.318676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320499-7d7f-4885-9340-cf6f0f290688 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:43.318909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320499-7d7f-4885-9340-cf6f0f290688 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:43.319391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320499-7d7f-4885-9340-cf6f0f290688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:43.350390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6c320499-7d7f-4885-9340-cf6f0f290688 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:43.351689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6c320499-7d7f-4885-9340-cf6f0f290688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:43.366251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 683/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Jun 30 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:43.372544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f6e9f-784f-4bad-b014-693250b263f0 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:43.388106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f6e9f-784f-4bad-b014-693250b263f0 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:43.393780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65921656-8a2b-478d-b410-272fc604a278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:43.394434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65921656-8a2b-478d-b410-272fc604a278 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:43.394713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65921656-8a2b-478d-b410-272fc604a278 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:43.394989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65921656-8a2b-478d-b410-272fc604a278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:43.404050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65921656-8a2b-478d-b410-272fc604a278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:43.405687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65921656-8a2b-478d-b410-272fc604a278 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:43.415421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 685/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:45.541646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300fc43-bc6c-4cfe-acf2-49a5383d13ff None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:45.542564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300fc43-bc6c-4cfe-acf2-49a5383d13ff None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:45.542808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300fc43-bc6c-4cfe-acf2-49a5383d13ff None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:45.544372 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 684/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:45.549898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7872862-51e7-4a66-94fd-cb705b9c9718 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:45.550360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7872862-51e7-4a66-94fd-cb705b9c9718 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:45.550475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7872862-51e7-4a66-94fd-cb705b9c9718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:45.569341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d7872862-51e7-4a66-94fd-cb705b9c9718 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:45.570537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d7872862-51e7-4a66-94fd-cb705b9c9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:45.580776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 686/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:45.625026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300fc43-bc6c-4cfe-acf2-49a5383d13ff None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:45.640795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300fc43-bc6c-4cfe-acf2-49a5383d13ff None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:45.646824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a62ddb-baa7-481d-a362-908f032c8051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:45.647524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a62ddb-baa7-481d-a362-908f032c8051 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:45.647797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a62ddb-baa7-481d-a362-908f032c8051 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:45.648068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a62ddb-baa7-481d-a362-908f032c8051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:45.657704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a62ddb-baa7-481d-a362-908f032c8051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:45.659324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a62ddb-baa7-481d-a362-908f032c8051 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:45.669341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 685/1371] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:46.406212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698170c-ab97-423c-b459-8b8cf2a84827 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:46.406708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698170c-ab97-423c-b459-8b8cf2a84827 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:46.407200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698170c-ab97-423c-b459-8b8cf2a84827 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:46.409029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 687/1372] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:46.414586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93963b1d-9c09-4a93-acb6-cef208d3b2fb None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:46.415123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93963b1d-9c09-4a93-acb6-cef208d3b2fb None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:46.415659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93963b1d-9c09-4a93-acb6-cef208d3b2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:46.436893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-93963b1d-9c09-4a93-acb6-cef208d3b2fb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:46.438067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-93963b1d-9c09-4a93-acb6-cef208d3b2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:46.448169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 686/1373] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:46.763105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88a09-c601-4089-be86-94d3a9f8c767 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:46.763649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88a09-c601-4089-be86-94d3a9f8c767 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:46.763649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88a09-c601-4089-be86-94d3a9f8c767 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:46.765021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 688/1374] 10.222.0.57 () {54 vars in 963 bytes} [Thu Jun 30 12:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:46.770186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ff9e-30de-493a-bfcd-ccbfb7fd8ecd None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:46.770970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ff9e-30de-493a-bfcd-ccbfb7fd8ecd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:46.771218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ff9e-30de-493a-bfcd-ccbfb7fd8ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:46.779485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:25:46.826482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d566ff9e-30de-493a-bfcd-ccbfb7fd8ecd None None] MFA Rules not processed for user `d61b8631879a461db96399a9b91682c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:46.827908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d566ff9e-30de-493a-bfcd-ccbfb7fd8ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:46.893332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 687/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Thu Jun 30 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:46.902719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88a09-c601-4089-be86-94d3a9f8c767 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:46.915269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88a09-c601-4089-be86-94d3a9f8c767 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:46.920157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1ed78-569a-4a07-a099-549fb353941e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:46.920953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1ed78-569a-4a07-a099-549fb353941e service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:46.921261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1ed78-569a-4a07-a099-549fb353941e service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:46.921539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1ed78-569a-4a07-a099-549fb353941e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:46.932658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1ed78-569a-4a07-a099-549fb353941e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:46.934869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1ed78-569a-4a07-a099-549fb353941e service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:46.959364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 689/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:47.770604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d931-724d-4201-a5d7-5959c36eda96 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:47.771158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d931-724d-4201-a5d7-5959c36eda96 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:47.771270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d931-724d-4201-a5d7-5959c36eda96 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:47.772810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 688/1377] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:47.779834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f35cb-0dfa-4334-b5e6-133da6838274 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:47.780286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f35cb-0dfa-4334-b5e6-133da6838274 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:47.780382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f35cb-0dfa-4334-b5e6-133da6838274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:47.802110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1d7f35cb-0dfa-4334-b5e6-133da6838274 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:47.803551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1d7f35cb-0dfa-4334-b5e6-133da6838274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:47.813273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 690/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:47.861736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55d931-724d-4201-a5d7-5959c36eda96 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:47.878015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55d931-724d-4201-a5d7-5959c36eda96 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:47.884606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae409fe7-a54e-4a1f-9020-00fee449f669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:47.885357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409fe7-a54e-4a1f-9020-00fee449f669 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:47.885703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409fe7-a54e-4a1f-9020-00fee449f669 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:47.885974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409fe7-a54e-4a1f-9020-00fee449f669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:47.894547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae409fe7-a54e-4a1f-9020-00fee449f669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:47.896169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae409fe7-a54e-4a1f-9020-00fee449f669 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:47.906108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 689/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:49.957361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1f00-92fd-4d0e-a7db-34fa1cf0c41b None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:49.958265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1f00-92fd-4d0e-a7db-34fa1cf0c41b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:49.958751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1f00-92fd-4d0e-a7db-34fa1cf0c41b None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:49.965906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6df6ab-17ad-490a-8a9a-e59d2601cd4f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:49.966742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6df6ab-17ad-490a-8a9a-e59d2601cd4f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:49.967228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6df6ab-17ad-490a-8a9a-e59d2601cd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:49.973016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 691/1380] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:49.988491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-1b6df6ab-17ad-490a-8a9a-e59d2601cd4f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:49.989726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1b6df6ab-17ad-490a-8a9a-e59d2601cd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:50.000704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 690/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:50.051359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb1f00-92fd-4d0e-a7db-34fa1cf0c41b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:50.066911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb1f00-92fd-4d0e-a7db-34fa1cf0c41b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:50.072286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923ac1e-8377-4f14-b98d-77e0d0099ca7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:50.072971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923ac1e-8377-4f14-b98d-77e0d0099ca7 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:50.073252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923ac1e-8377-4f14-b98d-77e0d0099ca7 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:50.073524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923ac1e-8377-4f14-b98d-77e0d0099ca7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:50.081624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923ac1e-8377-4f14-b98d-77e0d0099ca7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:50.083208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923ac1e-8377-4f14-b98d-77e0d0099ca7 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:50.093308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 692/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:52.161835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20082a67-7d01-4773-8a47-72609fa54966 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:52.162366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20082a67-7d01-4773-8a47-72609fa54966 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:52.162366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20082a67-7d01-4773-8a47-72609fa54966 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:52.167946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 691/1383] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:52.176683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab53d-6c26-4b8f-93eb-d5050c097dfa None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:52.177494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab53d-6c26-4b8f-93eb-d5050c097dfa None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:52.177781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab53d-6c26-4b8f-93eb-d5050c097dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:52.197686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6d8ab53d-6c26-4b8f-93eb-d5050c097dfa None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:52.199012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6d8ab53d-6c26-4b8f-93eb-d5050c097dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:52.208013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 693/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:52.249885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20082a67-7d01-4773-8a47-72609fa54966 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:52.266388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20082a67-7d01-4773-8a47-72609fa54966 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:52.272590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7d319-54e2-40fc-a095-fa748cb4a418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:52.273345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7d319-54e2-40fc-a095-fa748cb4a418 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:52.273629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7d319-54e2-40fc-a095-fa748cb4a418 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:52.273907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7d319-54e2-40fc-a095-fa748cb4a418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:52.283211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7d319-54e2-40fc-a095-fa748cb4a418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:52.284849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7d319-54e2-40fc-a095-fa748cb4a418 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:52.295580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 692/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:54.350892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29ff7c-4ff8-4da2-96cb-3a79f81327f0 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:54.350892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29ff7c-4ff8-4da2-96cb-3a79f81327f0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:54.351571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29ff7c-4ff8-4da2-96cb-3a79f81327f0 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:54.356476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 694/1386] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:54.357864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ece3d-fa6c-4bfa-8a2a-f10b7af32b52 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:54.358253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ece3d-fa6c-4bfa-8a2a-f10b7af32b52 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:54.358539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ece3d-fa6c-4bfa-8a2a-f10b7af32b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:54.389166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2c6ece3d-fa6c-4bfa-8a2a-f10b7af32b52 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:54.390318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2c6ece3d-fa6c-4bfa-8a2a-f10b7af32b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:54.423241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 693/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:25:54.455444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29ff7c-4ff8-4da2-96cb-3a79f81327f0 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:54.472482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29ff7c-4ff8-4da2-96cb-3a79f81327f0 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:54.478083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc647393-2a1e-4d0f-991c-551be7e2699c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:54.478737 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc647393-2a1e-4d0f-991c-551be7e2699c service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:54.479085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc647393-2a1e-4d0f-991c-551be7e2699c service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:54.479344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc647393-2a1e-4d0f-991c-551be7e2699c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:54.488870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc647393-2a1e-4d0f-991c-551be7e2699c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:54.490457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc647393-2a1e-4d0f-991c-551be7e2699c service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:54.501685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 695/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:56.749908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aa80c-6e17-45e4-a022-218436e0a667 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:56.750482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aa80c-6e17-45e4-a022-218436e0a667 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:56.750482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aa80c-6e17-45e4-a022-218436e0a667 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:56.752420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 694/1389] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:56.758558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac043e-c276-46b8-83d8-e778f0970e58 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:56.759148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac043e-c276-46b8-83d8-e778f0970e58 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:56.759230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac043e-c276-46b8-83d8-e778f0970e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:56.779857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-fcac043e-c276-46b8-83d8-e778f0970e58 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:56.781354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fcac043e-c276-46b8-83d8-e778f0970e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:56.790508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 696/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:25:56.843568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aa80c-6e17-45e4-a022-218436e0a667 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:56.861714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aa80c-6e17-45e4-a022-218436e0a667 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:56.873723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f329422-bdc5-43e2-aee0-d74b4da17429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:56.874416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329422-bdc5-43e2-aee0-d74b4da17429 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:56.874698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329422-bdc5-43e2-aee0-d74b4da17429 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:56.874959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f329422-bdc5-43e2-aee0-d74b4da17429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:56.884930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329422-bdc5-43e2-aee0-d74b4da17429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:56.886638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f329422-bdc5-43e2-aee0-d74b4da17429 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:56.896703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 695/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:25:59.599760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237df98e-d905-40a8-9ade-79ac8c67123a None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:59.600492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237df98e-d905-40a8-9ade-79ac8c67123a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:59.600492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237df98e-d905-40a8-9ade-79ac8c67123a None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:59.602425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 697/1392] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:25:59.611461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002c9f0-cacb-469d-ab86-e54e79f7eb93 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:59.611983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002c9f0-cacb-469d-ab86-e54e79f7eb93 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:59.612089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002c9f0-cacb-469d-ab86-e54e79f7eb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:59.632459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b002c9f0-cacb-469d-ab86-e54e79f7eb93 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:25:59.633771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b002c9f0-cacb-469d-ab86-e54e79f7eb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:25:59.643539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 696/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:25:59.695728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237df98e-d905-40a8-9ade-79ac8c67123a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:25:59.715866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237df98e-d905-40a8-9ade-79ac8c67123a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:25:59.723036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94ada2-894a-4c42-b7c9-29fe74743376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:25:59.723879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94ada2-894a-4c42-b7c9-29fe74743376 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:25:59.724537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94ada2-894a-4c42-b7c9-29fe74743376 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:25:59.724919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94ada2-894a-4c42-b7c9-29fe74743376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:25:59.736467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94ada2-894a-4c42-b7c9-29fe74743376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:25:59.738703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94ada2-894a-4c42-b7c9-29fe74743376 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:25:59.752051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 698/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:01.877736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabb7a4-2f02-4339-8cea-fcbdbb9126b9 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:01.878337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabb7a4-2f02-4339-8cea-fcbdbb9126b9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:01.878337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabb7a4-2f02-4339-8cea-fcbdbb9126b9 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:01.879671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 697/1395] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:01.885284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7136-d512-469d-bdc1-9aed5d67b8cc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:01.885707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7136-d512-469d-bdc1-9aed5d67b8cc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:01.885864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7136-d512-469d-bdc1-9aed5d67b8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:01.905113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a5ed7136-d512-469d-bdc1-9aed5d67b8cc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:01.906313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a5ed7136-d512-469d-bdc1-9aed5d67b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:01.914410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 699/1396] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:01.960276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabb7a4-2f02-4339-8cea-fcbdbb9126b9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:01.977158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabb7a4-2f02-4339-8cea-fcbdbb9126b9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:01.983285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c7fcb-0b7d-41fe-adb7-9689fc4396b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:01.983937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c7fcb-0b7d-41fe-adb7-9689fc4396b9 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:01.984215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c7fcb-0b7d-41fe-adb7-9689fc4396b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:01.984484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c7fcb-0b7d-41fe-adb7-9689fc4396b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:01.993565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c7fcb-0b7d-41fe-adb7-9689fc4396b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:01.995207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c7fcb-0b7d-41fe-adb7-9689fc4396b9 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:02.004753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 698/1397] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:02.387452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8363be1-a5a0-4661-8eba-c7c5cfa2fd0e None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:02.387719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8363be1-a5a0-4661-8eba-c7c5cfa2fd0e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:02.387998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8363be1-a5a0-4661-8eba-c7c5cfa2fd0e None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:02.389608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 700/1398] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:02.394767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f072f-2664-457a-b70d-76728700b3a7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:02.395083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f072f-2664-457a-b70d-76728700b3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:02.395361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f072f-2664-457a-b70d-76728700b3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:02.415778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d87f072f-2664-457a-b70d-76728700b3a7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:02.417068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d87f072f-2664-457a-b70d-76728700b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:02.425747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 699/1399] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:02.482296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d28d-8159-40f0-89b9-234d4ce02f90 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:02.482664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d28d-8159-40f0-89b9-234d4ce02f90 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:02.482980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d28d-8159-40f0-89b9-234d4ce02f90 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:02.484472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 701/1400] 10.222.0.57 () {54 vars in 962 bytes} [Thu Jun 30 12:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:02.490070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95decb-2c1a-4ea0-a770-ca2a85e9a004 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:02.490385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95decb-2c1a-4ea0-a770-ca2a85e9a004 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:02.490660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95decb-2c1a-4ea0-a770-ca2a85e9a004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:02.544928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2a95decb-2c1a-4ea0-a770-ca2a85e9a004 None None] MFA Rules not processed for user `a8e81a1c18d34adf8d8a45fa09f26099`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:02.546143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2a95decb-2c1a-4ea0-a770-ca2a85e9a004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:02.581972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:26:02.611500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 700/1401] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Jun 30 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:02.619227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129d28d-8159-40f0-89b9-234d4ce02f90 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:02.630031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129d28d-8159-40f0-89b9-234d4ce02f90 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:02.634136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7ab5e-64a4-4875-b8c4-f24785b8788c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:02.634800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ab5e-64a4-4875-b8c4-f24785b8788c service glance] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:02.635080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ab5e-64a4-4875-b8c4-f24785b8788c service glance] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:02.635354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ab5e-64a4-4875-b8c4-f24785b8788c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:02.644617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ab5e-64a4-4875-b8c4-f24785b8788c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:02.646135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ab5e-64a4-4875-b8c4-f24785b8788c service glance] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:02.656228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 702/1402] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:03.378074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4ba-cc1b-450e-bfce-d9ef3504da6c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:03.378566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4ba-cc1b-450e-bfce-d9ef3504da6c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:03.379361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4ba-cc1b-450e-bfce-d9ef3504da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:03.429218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7805b4ba-cc1b-450e-bfce-d9ef3504da6c None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:03.430433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7805b4ba-cc1b-450e-bfce-d9ef3504da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:03.502695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 701/1403] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:03.552023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a68e9f-6b10-48ad-b09b-cc681620bd46 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:03.552588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a68e9f-6b10-48ad-b09b-cc681620bd46 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:03.552588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a68e9f-6b10-48ad-b09b-cc681620bd46 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:03.554106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 703/1404] 10.222.0.57 () {54 vars in 942 bytes} [Thu Jun 30 12:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:03.561961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82739cd5-7e8d-4748-a974-b2bdbd4ce77b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:03.562267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82739cd5-7e8d-4748-a974-b2bdbd4ce77b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:03.562550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82739cd5-7e8d-4748-a974-b2bdbd4ce77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:03.612570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-82739cd5-7e8d-4748-a974-b2bdbd4ce77b None None] MFA Rules not processed for user `b96cbdec0f56412fb62617f03e77e713`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:03.613809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-82739cd5-7e8d-4748-a974-b2bdbd4ce77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:03.689650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 702/1405] 10.222.0.57 () {58 vars in 1039 bytes} [Thu Jun 30 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:26:03.699300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a68e9f-6b10-48ad-b09b-cc681620bd46 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:03.706927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=61 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:26:03.714240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a68e9f-6b10-48ad-b09b-cc681620bd46 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:03.719215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc1558-6e39-4709-8fbc-747658c97637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:03.719865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc1558-6e39-4709-8fbc-747658c97637 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:03.720148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc1558-6e39-4709-8fbc-747658c97637 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:03.720419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc1558-6e39-4709-8fbc-747658c97637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:03.728798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc1558-6e39-4709-8fbc-747658c97637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:03.730379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc1558-6e39-4709-8fbc-747658c97637 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:03.738562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 704/1406] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:26:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 12:26:04.097616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb9cb-806a-4377-9a79-6ca73a837756 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:04.097986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb9cb-806a-4377-9a79-6ca73a837756 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:04.098183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb9cb-806a-4377-9a79-6ca73a837756 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:04.099574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 703/1407] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:04.105291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c55a3-e964-497f-9ef2-1015cdbd19e1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:04.105571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c55a3-e964-497f-9ef2-1015cdbd19e1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:04.105881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c55a3-e964-497f-9ef2-1015cdbd19e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:04.128119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-434c55a3-e964-497f-9ef2-1015cdbd19e1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:04.129368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-434c55a3-e964-497f-9ef2-1015cdbd19e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:04.140674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 705/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:04.189256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eb9cb-806a-4377-9a79-6ca73a837756 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:04.204576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eb9cb-806a-4377-9a79-6ca73a837756 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:04.210220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3f464-1ddf-476e-98e5-21dffe995165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:04.210889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f464-1ddf-476e-98e5-21dffe995165 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:04.211159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f464-1ddf-476e-98e5-21dffe995165 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:04.211430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f464-1ddf-476e-98e5-21dffe995165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:04.220882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3f464-1ddf-476e-98e5-21dffe995165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:04.222491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3f464-1ddf-476e-98e5-21dffe995165 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:04.232169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:26:04.238123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 704/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:14.120799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=8 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:26:25.982100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824d3c-4adc-40c8-bf6e-0cbd9bc92d60 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:25.982710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824d3c-4adc-40c8-bf6e-0cbd9bc92d60 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:25.982710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824d3c-4adc-40c8-bf6e-0cbd9bc92d60 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:25.984097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 706/1410] 10.222.0.57 () {54 vars in 918 bytes} [Thu Jun 30 12:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:25.989458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63499-ed7d-42f6-938d-c4de311e6a42 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:25.989866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63499-ed7d-42f6-938d-c4de311e6a42 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:25.989998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63499-ed7d-42f6-938d-c4de311e6a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:26.009618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cbb63499-ed7d-42f6-938d-c4de311e6a42 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:26.010844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cbb63499-ed7d-42f6-938d-c4de311e6a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:26.021013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 705/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jun 30 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:26.035827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce53dd-eccf-4702-b628-93156ed20ebc None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:26.036117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce53dd-eccf-4702-b628-93156ed20ebc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:26.036397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce53dd-eccf-4702-b628-93156ed20ebc None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:26.037822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 707/1412] 10.222.0.57 () {54 vars in 918 bytes} [Thu Jun 30 12:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:26.042661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906aa1e-b5de-4975-abc5-f8aa15af46ef None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:26.042958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906aa1e-b5de-4975-abc5-f8aa15af46ef None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:26.043238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906aa1e-b5de-4975-abc5-f8aa15af46ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:26.063501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-3906aa1e-b5de-4975-abc5-f8aa15af46ef None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:26.064732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3906aa1e-b5de-4975-abc5-f8aa15af46ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:26.075000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 706/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jun 30 12:26:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:28.657193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512e34-0c01-4af8-b9cb-81e216bca578 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:28.658039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512e34-0c01-4af8-b9cb-81e216bca578 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:28.658039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512e34-0c01-4af8-b9cb-81e216bca578 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:28.660075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:28.665730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9e61-6b0c-4627-8735-01587c67dc7b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:28.666201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9e61-6b0c-4627-8735-01587c67dc7b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:28.666303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9e61-6b0c-4627-8735-01587c67dc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:28.685194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-79bc9e61-6b0c-4627-8735-01587c67dc7b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:28.686402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-79bc9e61-6b0c-4627-8735-01587c67dc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:28.694937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:28.742176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512e34-0c01-4af8-b9cb-81e216bca578 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:28.765749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512e34-0c01-4af8-b9cb-81e216bca578 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:28.771833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cebeb0-4d3e-4ba5-a4ba-848eeef4966b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:28.772482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cebeb0-4d3e-4ba5-a4ba-848eeef4966b service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:28.772792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cebeb0-4d3e-4ba5-a4ba-848eeef4966b service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:28.773068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cebeb0-4d3e-4ba5-a4ba-848eeef4966b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:28.782034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cebeb0-4d3e-4ba5-a4ba-848eeef4966b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:28.783628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cebeb0-4d3e-4ba5-a4ba-848eeef4966b service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:28.793358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 709/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:29.714952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507b1d5-cf9a-46f8-9f69-7704d961e323 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:29.715509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507b1d5-cf9a-46f8-9f69-7704d961e323 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:29.715509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507b1d5-cf9a-46f8-9f69-7704d961e323 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:29.717766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:29.726284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe807a-a151-4b38-8f62-9ca0c22c562e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:29.726702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe807a-a151-4b38-8f62-9ca0c22c562e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:29.726860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe807a-a151-4b38-8f62-9ca0c22c562e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:29.753060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-73fe807a-a151-4b38-8f62-9ca0c22c562e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:29.754272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-73fe807a-a151-4b38-8f62-9ca0c22c562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:29.764560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:29.926122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507b1d5-cf9a-46f8-9f69-7704d961e323 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:29.944176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507b1d5-cf9a-46f8-9f69-7704d961e323 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:29.950779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe341e-edee-4fdc-a7f2-5486c11c12a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:29.951457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe341e-edee-4fdc-a7f2-5486c11c12a7 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:29.951745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe341e-edee-4fdc-a7f2-5486c11c12a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:29.952016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe341e-edee-4fdc-a7f2-5486c11c12a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:29.960801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe341e-edee-4fdc-a7f2-5486c11c12a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:29.962883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe341e-edee-4fdc-a7f2-5486c11c12a7 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:29.972882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 709/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:30.861427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36981e-7bce-4c05-a642-7fa8bb539e16 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:30.862037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36981e-7bce-4c05-a642-7fa8bb539e16 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:30.862037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36981e-7bce-4c05-a642-7fa8bb539e16 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:30.863397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:30.869571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ac9f-379e-4578-b902-46b99611fbaa None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:30.870310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ac9f-379e-4578-b902-46b99611fbaa None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:30.870468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ac9f-379e-4578-b902-46b99611fbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:30.895492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-85b4ac9f-379e-4578-b902-46b99611fbaa None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:30.896736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-85b4ac9f-379e-4578-b902-46b99611fbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:30.905771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:31.061941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36981e-7bce-4c05-a642-7fa8bb539e16 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:31.081487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36981e-7bce-4c05-a642-7fa8bb539e16 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:31.087944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf4026-0ece-454b-b434-9ec6df98f026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:31.088595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4026-0ece-454b-b434-9ec6df98f026 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:31.088910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4026-0ece-454b-b434-9ec6df98f026 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:31.089179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4026-0ece-454b-b434-9ec6df98f026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:31.097640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf4026-0ece-454b-b434-9ec6df98f026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:31.099217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf4026-0ece-454b-b434-9ec6df98f026 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:31.109100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 712/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:32.033781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54934ec-2271-49e4-ae5a-18394c09fb30 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:32.034327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54934ec-2271-49e4-ae5a-18394c09fb30 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:32.034327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54934ec-2271-49e4-ae5a-18394c09fb30 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:32.035698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:32.041296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53497ab-847e-4b64-96e4-da7730366061 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:32.041745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53497ab-847e-4b64-96e4-da7730366061 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:32.041867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53497ab-847e-4b64-96e4-da7730366061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:32.063069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e53497ab-847e-4b64-96e4-da7730366061 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:32.064266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e53497ab-847e-4b64-96e4-da7730366061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:32.077984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:32.261599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54934ec-2271-49e4-ae5a-18394c09fb30 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:32.287657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54934ec-2271-49e4-ae5a-18394c09fb30 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:32.295270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187da686-f3fd-47a1-9252-59e60e393e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:32.295919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187da686-f3fd-47a1-9252-59e60e393e94 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:32.296202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187da686-f3fd-47a1-9252-59e60e393e94 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:32.296477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187da686-f3fd-47a1-9252-59e60e393e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:32.305660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187da686-f3fd-47a1-9252-59e60e393e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:32.307634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187da686-f3fd-47a1-9252-59e60e393e94 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:32.317643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 712/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:33.190393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb78bb-f94f-4330-9dfd-b8fe161a252f None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:33.190902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb78bb-f94f-4330-9dfd-b8fe161a252f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:33.190902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb78bb-f94f-4330-9dfd-b8fe161a252f None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:33.192343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:33.197898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e35fb-5911-4f80-9301-103e650c03fc None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:33.198474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e35fb-5911-4f80-9301-103e650c03fc None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:33.198474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e35fb-5911-4f80-9301-103e650c03fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:33.217193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-333e35fb-5911-4f80-9301-103e650c03fc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:33.218413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-333e35fb-5911-4f80-9301-103e650c03fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:33.226951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:33.376661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb78bb-f94f-4330-9dfd-b8fe161a252f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:33.392584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb78bb-f94f-4330-9dfd-b8fe161a252f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:33.398560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd711fcf-0c09-4429-929b-c5f11f53ff66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:33.399206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd711fcf-0c09-4429-929b-c5f11f53ff66 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:33.399485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd711fcf-0c09-4429-929b-c5f11f53ff66 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:33.400028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd711fcf-0c09-4429-929b-c5f11f53ff66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:33.408488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd711fcf-0c09-4429-929b-c5f11f53ff66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:33.410108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd711fcf-0c09-4429-929b-c5f11f53ff66 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:33.420705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 715/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:34.262073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc97311-9684-424e-ad7d-9ee6417a84bd None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:34.262638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc97311-9684-424e-ad7d-9ee6417a84bd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:34.262638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc97311-9684-424e-ad7d-9ee6417a84bd None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:34.263917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:34.270302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd8e22-1a69-4c7b-8693-893e221c52a6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:34.271065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd8e22-1a69-4c7b-8693-893e221c52a6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:34.271252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd8e22-1a69-4c7b-8693-893e221c52a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:34.291328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-44fd8e22-1a69-4c7b-8693-893e221c52a6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:34.292551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-44fd8e22-1a69-4c7b-8693-893e221c52a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:34.302128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:34.459998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc97311-9684-424e-ad7d-9ee6417a84bd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:34.475953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc97311-9684-424e-ad7d-9ee6417a84bd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:34.481960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ca435-d270-42db-89c2-802c9f7b5aec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:34.482607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ca435-d270-42db-89c2-802c9f7b5aec service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:34.482890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ca435-d270-42db-89c2-802c9f7b5aec service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:34.483165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ca435-d270-42db-89c2-802c9f7b5aec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:34.491209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ca435-d270-42db-89c2-802c9f7b5aec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:34.492841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ca435-d270-42db-89c2-802c9f7b5aec service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:34.502292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 715/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:35.309282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9924-ded9-4db6-a6c2-ed446b875193 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:35.309893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9924-ded9-4db6-a6c2-ed446b875193 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:35.309893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9924-ded9-4db6-a6c2-ed446b875193 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:35.311229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:35.317409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3746-0861-4f6f-bf27-47ef3035a9b4 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:35.318104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3746-0861-4f6f-bf27-47ef3035a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:35.318310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3746-0861-4f6f-bf27-47ef3035a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:35.340197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-049a3746-0861-4f6f-bf27-47ef3035a9b4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:35.341450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-049a3746-0861-4f6f-bf27-47ef3035a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:35.353981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:35.514410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec9924-ded9-4db6-a6c2-ed446b875193 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:35.537721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec9924-ded9-4db6-a6c2-ed446b875193 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:35.544464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8b3e2-3161-4abb-a380-71f5115dd3b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:35.545142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b3e2-3161-4abb-a380-71f5115dd3b4 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:35.545421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b3e2-3161-4abb-a380-71f5115dd3b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:35.545689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b3e2-3161-4abb-a380-71f5115dd3b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:35.554311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8b3e2-3161-4abb-a380-71f5115dd3b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:35.556009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8b3e2-3161-4abb-a380-71f5115dd3b4 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:35.567133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 718/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:36.380966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8074f0-b8b4-4796-bf41-eb676c42ec3f None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:36.381534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8074f0-b8b4-4796-bf41-eb676c42ec3f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:36.381534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8074f0-b8b4-4796-bf41-eb676c42ec3f None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:36.382802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:36.388668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0c53-4a5e-40aa-ab16-7a8b7b8c215d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:36.389098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0c53-4a5e-40aa-ab16-7a8b7b8c215d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:36.389302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0c53-4a5e-40aa-ab16-7a8b7b8c215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:36.408996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8cbc0c53-4a5e-40aa-ab16-7a8b7b8c215d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:36.410204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8cbc0c53-4a5e-40aa-ab16-7a8b7b8c215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:36.419453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:36.576121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8074f0-b8b4-4796-bf41-eb676c42ec3f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:36.596282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8074f0-b8b4-4796-bf41-eb676c42ec3f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:36.602144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac4e31-8db9-4efc-8e4e-709cd272b25e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:36.602959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4e31-8db9-4efc-8e4e-709cd272b25e service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:36.603201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4e31-8db9-4efc-8e4e-709cd272b25e service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:36.603363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4e31-8db9-4efc-8e4e-709cd272b25e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:36.611714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac4e31-8db9-4efc-8e4e-709cd272b25e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:36.613349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac4e31-8db9-4efc-8e4e-709cd272b25e service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:36.622943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 718/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:37.420662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035a0eb-908f-442b-ad30-b4d265b58bd5 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:37.421229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035a0eb-908f-442b-ad30-b4d265b58bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:37.421229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035a0eb-908f-442b-ad30-b4d265b58bd5 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:37.422653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:37.428994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe27b0d-41b0-4b51-8e80-acd57e814271 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:37.429735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe27b0d-41b0-4b51-8e80-acd57e814271 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:37.429880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe27b0d-41b0-4b51-8e80-acd57e814271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:37.452191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2fe27b0d-41b0-4b51-8e80-acd57e814271 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:37.453454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2fe27b0d-41b0-4b51-8e80-acd57e814271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:37.462468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:37.614774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035a0eb-908f-442b-ad30-b4d265b58bd5 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:37.638503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035a0eb-908f-442b-ad30-b4d265b58bd5 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:37.644566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e423ec-9f4a-48fa-9392-5964c0a28455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:37.645244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e423ec-9f4a-48fa-9392-5964c0a28455 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:37.645528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e423ec-9f4a-48fa-9392-5964c0a28455 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:37.645801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e423ec-9f4a-48fa-9392-5964c0a28455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:37.654114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e423ec-9f4a-48fa-9392-5964c0a28455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:37.655716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e423ec-9f4a-48fa-9392-5964c0a28455 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:37.665502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 721/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:39.605071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c501c6-004d-46c5-b9d8-47d902c2f472 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:39.606169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c501c6-004d-46c5-b9d8-47d902c2f472 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:39.606169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c501c6-004d-46c5-b9d8-47d902c2f472 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:39.608116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:39.614663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee83ee2-a100-4c18-8db8-f46206c1f8c5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:39.615129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee83ee2-a100-4c18-8db8-f46206c1f8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:39.615239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee83ee2-a100-4c18-8db8-f46206c1f8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:39.634847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-dee83ee2-a100-4c18-8db8-f46206c1f8c5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:39.636054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dee83ee2-a100-4c18-8db8-f46206c1f8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:39.644660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:39.685894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c501c6-004d-46c5-b9d8-47d902c2f472 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:39.702361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c501c6-004d-46c5-b9d8-47d902c2f472 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:39.706531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c9ad9-315a-4a75-ab96-3e047dcc3c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:39.707178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9ad9-315a-4a75-ab96-3e047dcc3c29 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:39.707438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9ad9-315a-4a75-ab96-3e047dcc3c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:39.707721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9ad9-315a-4a75-ab96-3e047dcc3c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:39.716937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c9ad9-315a-4a75-ab96-3e047dcc3c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:39.718382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c9ad9-315a-4a75-ab96-3e047dcc3c29 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:39.728065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 721/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:41.967702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085c50-d119-4919-9a48-9a5ae632763e None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:41.968478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085c50-d119-4919-9a48-9a5ae632763e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:41.968687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085c50-d119-4919-9a48-9a5ae632763e None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:41.970903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:41.977208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22e3a-8e5d-4082-85dd-b39101a8c283 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:41.978115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22e3a-8e5d-4082-85dd-b39101a8c283 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:41.978238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22e3a-8e5d-4082-85dd-b39101a8c283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:41.996680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6ba22e3a-8e5d-4082-85dd-b39101a8c283 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:41.997919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6ba22e3a-8e5d-4082-85dd-b39101a8c283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:42.006222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:42.045696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085c50-d119-4919-9a48-9a5ae632763e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:42.048582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-87085c50-d119-4919-9a48-9a5ae632763e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:42.070843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085c50-d119-4919-9a48-9a5ae632763e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:42.075421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac52ddd-8f38-4a59-9333-ca6ac6dfdc41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:42.076069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52ddd-8f38-4a59-9333-ca6ac6dfdc41 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:42.076350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52ddd-8f38-4a59-9333-ca6ac6dfdc41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:42.076641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52ddd-8f38-4a59-9333-ca6ac6dfdc41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:42.087081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac52ddd-8f38-4a59-9333-ca6ac6dfdc41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:42.088654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac52ddd-8f38-4a59-9333-ca6ac6dfdc41 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:42.098522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 724/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:44.488821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48abd6-09d9-4e78-b038-a6d7531e67f3 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:44.489344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48abd6-09d9-4e78-b038-a6d7531e67f3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:44.489344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48abd6-09d9-4e78-b038-a6d7531e67f3 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:44.491587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:44.498017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0b8cf-c984-46e9-81e6-155396ca0ae0 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:44.498490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0b8cf-c984-46e9-81e6-155396ca0ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:44.498595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0b8cf-c984-46e9-81e6-155396ca0ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:44.526129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-11d0b8cf-c984-46e9-81e6-155396ca0ae0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:44.527368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-11d0b8cf-c984-46e9-81e6-155396ca0ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:44.536657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:44.576419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48abd6-09d9-4e78-b038-a6d7531e67f3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:44.588584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48abd6-09d9-4e78-b038-a6d7531e67f3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:44.592358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdb5a2-21c5-4f26-af25-c397f085f703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:44.593031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb5a2-21c5-4f26-af25-c397f085f703 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:44.593305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb5a2-21c5-4f26-af25-c397f085f703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:44.593575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb5a2-21c5-4f26-af25-c397f085f703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:44.602804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdb5a2-21c5-4f26-af25-c397f085f703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:44.604313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdb5a2-21c5-4f26-af25-c397f085f703 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:44.613752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 724/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:46.770956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e42a6-6b73-4945-b34c-8c4f77eaee73 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:46.771570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e42a6-6b73-4945-b34c-8c4f77eaee73 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:46.771570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e42a6-6b73-4945-b34c-8c4f77eaee73 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:46.772970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:46.778651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43177b-67f9-4bd2-913c-c6c7348684dc None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:46.779123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43177b-67f9-4bd2-913c-c6c7348684dc None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:46.779241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43177b-67f9-4bd2-913c-c6c7348684dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:46.797709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-4e43177b-67f9-4bd2-913c-c6c7348684dc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:46.798921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4e43177b-67f9-4bd2-913c-c6c7348684dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:46.809124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:46.853733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e42a6-6b73-4945-b34c-8c4f77eaee73 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:46.866138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e42a6-6b73-4945-b34c-8c4f77eaee73 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:46.870338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85109e75-3f28-40a2-bddb-47367a292e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:46.870957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85109e75-3f28-40a2-bddb-47367a292e8b service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:46.871255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85109e75-3f28-40a2-bddb-47367a292e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:46.871508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85109e75-3f28-40a2-bddb-47367a292e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:46.880987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85109e75-3f28-40a2-bddb-47367a292e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:46.882392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85109e75-3f28-40a2-bddb-47367a292e8b service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:46.892025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 727/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:49.149637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51531bfb-956a-4376-bc33-a749934d8b75 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:49.150253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51531bfb-956a-4376-bc33-a749934d8b75 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:49.150253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51531bfb-956a-4376-bc33-a749934d8b75 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:49.152047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:49.160560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced98acd-aa60-4215-9eea-d972d9fe96bc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:49.161027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced98acd-aa60-4215-9eea-d972d9fe96bc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:49.161131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced98acd-aa60-4215-9eea-d972d9fe96bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:49.182286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ced98acd-aa60-4215-9eea-d972d9fe96bc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:49.183595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ced98acd-aa60-4215-9eea-d972d9fe96bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:49.193228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:49.233189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51531bfb-956a-4376-bc33-a749934d8b75 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:49.252517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51531bfb-956a-4376-bc33-a749934d8b75 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:49.259044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9073a6-b5a4-4cc9-84a4-4f10e8ac68c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:49.260046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9073a6-b5a4-4cc9-84a4-4f10e8ac68c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:49.260323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9073a6-b5a4-4cc9-84a4-4f10e8ac68c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:49.260595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9073a6-b5a4-4cc9-84a4-4f10e8ac68c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:49.272536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9073a6-b5a4-4cc9-84a4-4f10e8ac68c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:49.274086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9073a6-b5a4-4cc9-84a4-4f10e8ac68c4 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:49.284196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 727/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:51.413242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841c72d-f56f-4799-b760-510f2e0d60fe None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:51.413870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841c72d-f56f-4799-b760-510f2e0d60fe None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:51.413870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841c72d-f56f-4799-b760-510f2e0d60fe None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:51.415215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:51.420862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c48a3a-d3f2-438d-a139-74b4f99de6f8 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:51.421332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c48a3a-d3f2-438d-a139-74b4f99de6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:51.421438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c48a3a-d3f2-438d-a139-74b4f99de6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:51.441379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-57c48a3a-d3f2-438d-a139-74b4f99de6f8 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:51.442591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-57c48a3a-d3f2-438d-a139-74b4f99de6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:51.452017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:51.493269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841c72d-f56f-4799-b760-510f2e0d60fe None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:51.506701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841c72d-f56f-4799-b760-510f2e0d60fe None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:51.511645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4d6f-0d1a-4cbf-ad07-7c2db6471747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:51.512286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4d6f-0d1a-4cbf-ad07-7c2db6471747 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:51.512561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4d6f-0d1a-4cbf-ad07-7c2db6471747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:51.512862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4d6f-0d1a-4cbf-ad07-7c2db6471747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:51.523104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df4d6f-0d1a-4cbf-ad07-7c2db6471747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:51.524584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df4d6f-0d1a-4cbf-ad07-7c2db6471747 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:51.534397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 730/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:53.615778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9be39-c6d4-4865-a6fb-d7bbee23ce81 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:53.616410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9be39-c6d4-4865-a6fb-d7bbee23ce81 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:53.616410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9be39-c6d4-4865-a6fb-d7bbee23ce81 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:53.617750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:53.624350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542ccea-d679-4630-b083-1647b0678357 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:53.625192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542ccea-d679-4630-b083-1647b0678357 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:53.625356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542ccea-d679-4630-b083-1647b0678357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:53.648138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0542ccea-d679-4630-b083-1647b0678357 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:53.649364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0542ccea-d679-4630-b083-1647b0678357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:53.659226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:53.697854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9be39-c6d4-4865-a6fb-d7bbee23ce81 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:53.709967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9be39-c6d4-4865-a6fb-d7bbee23ce81 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:53.714549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9f420-bbbd-4db2-97de-f813b979f0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:53.715195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9f420-bbbd-4db2-97de-f813b979f0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:53.715509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9f420-bbbd-4db2-97de-f813b979f0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:53.715782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9f420-bbbd-4db2-97de-f813b979f0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:53.725035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9f420-bbbd-4db2-97de-f813b979f0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:53.726486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9f420-bbbd-4db2-97de-f813b979f0b3 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:53.736196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 730/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:55.869123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec35f-faa4-477b-a4d4-ee3b6655f311 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:55.869727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec35f-faa4-477b-a4d4-ee3b6655f311 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:55.869727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec35f-faa4-477b-a4d4-ee3b6655f311 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:55.871122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:55.877594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2a64c-972b-48c5-8097-05a10a44729a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:55.878357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2a64c-972b-48c5-8097-05a10a44729a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:55.878514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2a64c-972b-48c5-8097-05a10a44729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:55.887661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=79 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:26:55.902054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-96a2a64c-972b-48c5-8097-05a10a44729a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:55.903261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-96a2a64c-972b-48c5-8097-05a10a44729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:55.912578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:55.953388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec35f-faa4-477b-a4d4-ee3b6655f311 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:55.962041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=73 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:26:55.971631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec35f-faa4-477b-a4d4-ee3b6655f311 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:55.977692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280111b8-bb24-4cec-896b-79d3f2dfefa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:55.978702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280111b8-bb24-4cec-896b-79d3f2dfefa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:55.979166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280111b8-bb24-4cec-896b-79d3f2dfefa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:55.979517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280111b8-bb24-4cec-896b-79d3f2dfefa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:55.990320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280111b8-bb24-4cec-896b-79d3f2dfefa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:55.991762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280111b8-bb24-4cec-896b-79d3f2dfefa0 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:56.001096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 733/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:26:58.059177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef20150-0179-4569-acd9-de02d310cdf9 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:58.059978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef20150-0179-4569-acd9-de02d310cdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:58.059978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef20150-0179-4569-acd9-de02d310cdf9 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:58.062203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:26:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:26:58.068522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86789c7f-f1e1-4bfd-8e4e-96c702b51c78 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:58.069023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86789c7f-f1e1-4bfd-8e4e-96c702b51c78 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:58.069135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86789c7f-f1e1-4bfd-8e4e-96c702b51c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:58.088467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-86789c7f-f1e1-4bfd-8e4e-96c702b51c78 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:26:58.089707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-86789c7f-f1e1-4bfd-8e4e-96c702b51c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:26:58.098153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:26:58.134550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef20150-0179-4569-acd9-de02d310cdf9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:26:58.146707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef20150-0179-4569-acd9-de02d310cdf9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:26:58.150756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef1881-189b-4b91-886b-4e8e5aa0b49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:26:58.151397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef1881-189b-4b91-886b-4e8e5aa0b49b service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:26:58.151674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef1881-189b-4b91-886b-4e8e5aa0b49b service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:26:58.151953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef1881-189b-4b91-886b-4e8e5aa0b49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:26:58.161688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef1881-189b-4b91-886b-4e8e5aa0b49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:26:58.163128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef1881-189b-4b91-886b-4e8e5aa0b49b service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:26:58.173314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 733/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:27:00.233283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8190f8-2ed6-4c03-a792-2f81174462c6 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:00.233844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8190f8-2ed6-4c03-a792-2f81174462c6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:00.233844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8190f8-2ed6-4c03-a792-2f81174462c6 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:00.235152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:00.241862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656f84-1b9d-462f-960e-9e936850f63d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:00.242571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656f84-1b9d-462f-960e-9e936850f63d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:00.242721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656f84-1b9d-462f-960e-9e936850f63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:00.266537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-aa656f84-1b9d-462f-960e-9e936850f63d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:00.267752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-aa656f84-1b9d-462f-960e-9e936850f63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:00.277243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:00.316679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8190f8-2ed6-4c03-a792-2f81174462c6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:00.330043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8190f8-2ed6-4c03-a792-2f81174462c6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:00.335032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751228b0-8900-44fd-a189-0e82b2ff358a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:00.335707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751228b0-8900-44fd-a189-0e82b2ff358a service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:00.335986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751228b0-8900-44fd-a189-0e82b2ff358a service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:00.336260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751228b0-8900-44fd-a189-0e82b2ff358a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:00.346108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751228b0-8900-44fd-a189-0e82b2ff358a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:00.347572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751228b0-8900-44fd-a189-0e82b2ff358a service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:00.357403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 736/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:10.260671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=11 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:27:10.352843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=14 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:27:20.356210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230539-f2f5-4b28-9326-49042e8cdc42 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:20.356872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230539-f2f5-4b28-9326-49042e8cdc42 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:20.356872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d230539-f2f5-4b28-9326-49042e8cdc42 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:20.358173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 735/1471] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:20.365719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811ec8-4de8-4efb-9c85-7c79633225e3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:20.366433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811ec8-4de8-4efb-9c85-7c79633225e3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:20.366585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15811ec8-4de8-4efb-9c85-7c79633225e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:20.393810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-15811ec8-4de8-4efb-9c85-7c79633225e3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:20.395029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-15811ec8-4de8-4efb-9c85-7c79633225e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:20.405585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 737/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:20.458936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14b1b5-8cb4-4354-b23c-7d2440205071 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:20.459222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14b1b5-8cb4-4354-b23c-7d2440205071 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:20.459495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14b1b5-8cb4-4354-b23c-7d2440205071 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:20.460911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 736/1473] 10.222.0.57 () {54 vars in 962 bytes} [Thu Jun 30 12:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:20.465883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a990cd-47cd-4c14-89e1-1625b4139c00 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:20.466465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a990cd-47cd-4c14-89e1-1625b4139c00 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:20.466753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a990cd-47cd-4c14-89e1-1625b4139c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:20.486572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a0a990cd-47cd-4c14-89e1-1625b4139c00 None None] MFA Rules not processed for user `a8e81a1c18d34adf8d8a45fa09f26099`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:20.487800 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a0a990cd-47cd-4c14-89e1-1625b4139c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:20.495566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 738/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Jun 30 12:27:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:20.502910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14b1b5-8cb4-4354-b23c-7d2440205071 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:20.515239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14b1b5-8cb4-4354-b23c-7d2440205071 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:20.519317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5041513-8b17-4c68-8ea5-be025ef199fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:20.519961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5041513-8b17-4c68-8ea5-be025ef199fe service glance] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:20.520242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5041513-8b17-4c68-8ea5-be025ef199fe service glance] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:20.520517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5041513-8b17-4c68-8ea5-be025ef199fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:20.530105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5041513-8b17-4c68-8ea5-be025ef199fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:20.531556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5041513-8b17-4c68-8ea5-be025ef199fe service glance] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:20.541179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 737/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:22.691973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d77008-fe0b-4aaf-89fc-97b3deab68a2 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:22.692961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d77008-fe0b-4aaf-89fc-97b3deab68a2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:22.692961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d77008-fe0b-4aaf-89fc-97b3deab68a2 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:22.695196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 739/1476] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:22.700817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596c91c-df11-44a3-8e55-79df65b7162b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:22.701282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596c91c-df11-44a3-8e55-79df65b7162b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:22.701379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596c91c-df11-44a3-8e55-79df65b7162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:22.720414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8596c91c-df11-44a3-8e55-79df65b7162b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:22.721694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8596c91c-df11-44a3-8e55-79df65b7162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:22.734350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 738/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:22.757780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d77008-fe0b-4aaf-89fc-97b3deab68a2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:22.771540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d77008-fe0b-4aaf-89fc-97b3deab68a2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:22.777196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:22.777845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:22.778122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:22.778385 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:22.779112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:22.780884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:27:22.780884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:27:22.780884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:27:22.781986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec8b2e-3545-40c8-9f61-565baeeeae2c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:22.801872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 740/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jun 30 12:27:22 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:27:24.880027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b6eb1-54cf-4fff-bbe9-9e038776cf87 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:24.880883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b6eb1-54cf-4fff-bbe9-9e038776cf87 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:24.880883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b6eb1-54cf-4fff-bbe9-9e038776cf87 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:24.882953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 739/1479] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:24.889869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94eae3-ec6f-47e7-8c40-3feec8810606 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:24.890330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94eae3-ec6f-47e7-8c40-3feec8810606 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:24.890437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94eae3-ec6f-47e7-8c40-3feec8810606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:24.909534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4f94eae3-ec6f-47e7-8c40-3feec8810606 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:24.910752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4f94eae3-ec6f-47e7-8c40-3feec8810606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:24.919337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 741/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:24.968167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b6eb1-54cf-4fff-bbe9-9e038776cf87 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:24.991307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b6eb1-54cf-4fff-bbe9-9e038776cf87 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:24.997583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f035aa-13d8-410d-8ee1-bb049e21f035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:24.998519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f035aa-13d8-410d-8ee1-bb049e21f035 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:24.998800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f035aa-13d8-410d-8ee1-bb049e21f035 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:24.999076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f035aa-13d8-410d-8ee1-bb049e21f035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:25.007707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f035aa-13d8-410d-8ee1-bb049e21f035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:25.009865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f035aa-13d8-410d-8ee1-bb049e21f035 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:25.019602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 740/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:27.283852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6276c5-3524-46aa-b770-056daba2e7ca None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:27.284761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6276c5-3524-46aa-b770-056daba2e7ca None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:27.284761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6276c5-3524-46aa-b770-056daba2e7ca None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:27.287104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 742/1482] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:27.294914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b421e-3686-4770-8b7e-b7cfe29ade44 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:27.295338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b421e-3686-4770-8b7e-b7cfe29ade44 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:27.295450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b421e-3686-4770-8b7e-b7cfe29ade44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:27.315674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bf8b421e-3686-4770-8b7e-b7cfe29ade44 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:27.316941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bf8b421e-3686-4770-8b7e-b7cfe29ade44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:27.326307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 741/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:27.362787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6276c5-3524-46aa-b770-056daba2e7ca None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:27.375616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6276c5-3524-46aa-b770-056daba2e7ca None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:27.379670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6d85d-a5b6-4d9a-ab5e-cbf868286d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:27.380307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6d85d-a5b6-4d9a-ab5e-cbf868286d04 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:27.380588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6d85d-a5b6-4d9a-ab5e-cbf868286d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:27.380891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6d85d-a5b6-4d9a-ab5e-cbf868286d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:27.390903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6d85d-a5b6-4d9a-ab5e-cbf868286d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:27.392344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6d85d-a5b6-4d9a-ab5e-cbf868286d04 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:27.402911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 743/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:29.488748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33261743-447f-43e8-8ceb-94a5ffa500e6 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:29.489340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33261743-447f-43e8-8ceb-94a5ffa500e6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:29.489340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33261743-447f-43e8-8ceb-94a5ffa500e6 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:29.491429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 742/1485] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:29.500496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a83e70-f126-46a6-979d-7d8fe9ec9bf1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:29.501012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a83e70-f126-46a6-979d-7d8fe9ec9bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:29.501125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a83e70-f126-46a6-979d-7d8fe9ec9bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:29.525112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-96a83e70-f126-46a6-979d-7d8fe9ec9bf1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:29.526345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-96a83e70-f126-46a6-979d-7d8fe9ec9bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:29.535537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 744/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:29.574048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33261743-447f-43e8-8ceb-94a5ffa500e6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:29.585661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33261743-447f-43e8-8ceb-94a5ffa500e6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:29.589451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf809b6-7fbd-458c-ad85-2afea3e69a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:29.590118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf809b6-7fbd-458c-ad85-2afea3e69a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:29.590399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf809b6-7fbd-458c-ad85-2afea3e69a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:29.590671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf809b6-7fbd-458c-ad85-2afea3e69a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:29.600768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf809b6-7fbd-458c-ad85-2afea3e69a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:29.602228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf809b6-7fbd-458c-ad85-2afea3e69a0e service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:29.611902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 743/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:31.814699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c60b-4e2a-4fc2-9c0c-972b7d539c36 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:31.815308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c60b-4e2a-4fc2-9c0c-972b7d539c36 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:31.815308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c60b-4e2a-4fc2-9c0c-972b7d539c36 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:31.816658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 745/1488] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:31.823181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ef207-e837-4f57-a08a-e7d7a877eec6 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:31.824032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ef207-e837-4f57-a08a-e7d7a877eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:31.824346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ef207-e837-4f57-a08a-e7d7a877eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:31.853088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a66ef207-e837-4f57-a08a-e7d7a877eec6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:31.854867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a66ef207-e837-4f57-a08a-e7d7a877eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:31.863398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 744/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:31.909261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c60b-4e2a-4fc2-9c0c-972b7d539c36 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:31.921865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c60b-4e2a-4fc2-9c0c-972b7d539c36 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:31.926196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9f2c-d872-4a47-9ea3-10b7d561b2b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:31.926838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9f2c-d872-4a47-9ea3-10b7d561b2b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:31.927116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9f2c-d872-4a47-9ea3-10b7d561b2b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:31.927386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9f2c-d872-4a47-9ea3-10b7d561b2b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:31.937213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc9f2c-d872-4a47-9ea3-10b7d561b2b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:31.938655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc9f2c-d872-4a47-9ea3-10b7d561b2b4 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:31.947987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 746/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:34.247634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f9740-90bc-46e8-a078-792a8edb02c3 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:34.248469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f9740-90bc-46e8-a078-792a8edb02c3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:34.248469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f9740-90bc-46e8-a078-792a8edb02c3 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:34.250722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 745/1491] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jun 30 12:27:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:27:34.257214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33880-9954-4a3c-bc55-f83521cd2ebd None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:34.257924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33880-9954-4a3c-bc55-f83521cd2ebd None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:34.258070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33880-9954-4a3c-bc55-f83521cd2ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:34.283669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-60c33880-9954-4a3c-bc55-f83521cd2ebd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:27:34.284902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-60c33880-9954-4a3c-bc55-f83521cd2ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:27:34.292602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 747/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jun 30 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:27:34.332014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f9740-90bc-46e8-a078-792a8edb02c3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:27:34.344417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f9740-90bc-46e8-a078-792a8edb02c3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:27:34.348888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011bd4d-aebc-49bf-870a-2d0c039bda91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:27:34.349539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011bd4d-aebc-49bf-870a-2d0c039bda91 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:27:34.349820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011bd4d-aebc-49bf-870a-2d0c039bda91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:27:34.350096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011bd4d-aebc-49bf-870a-2d0c039bda91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:27:34.359346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011bd4d-aebc-49bf-870a-2d0c039bda91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:27:34.360829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011bd4d-aebc-49bf-870a-2d0c039bda91 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:27:34.370460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 746/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:27:44.278301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=33 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:27:44.365981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=33 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:29:01.314527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.315484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.315484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.336522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:29:01.337739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:01.346677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 748/1494] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:29:01.355778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.356429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.356429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.379822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:29:01.381087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:01.390614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 747/1495] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:29:01.395322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.405436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd6bf0-9a3c-4511-8640-24b87e886f95 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.410690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.411358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.411637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.411942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.412703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:29:01.413013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:01.414047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:01.414047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:29:01.414047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:29:01.414988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:01.427585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 749/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:29:01.437559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.447440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662dc9d-7f28-41d7-8356-d32d40e38b47 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.452367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.453067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.453351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.453621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.454319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:01.456163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:01.456163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:29:01.456163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:29:01.457402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:01.504351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 748/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:29:01 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:29:01.508828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.519286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ecd2a-514a-4d93-9dc4-b9a824125b8e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.581293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.582051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.582232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.582480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.583161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:01.584196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:01.584196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:29:01.584196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:29:01.585433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:01.610870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 750/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:29:01 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:29:01.615199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.627606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0612f-3eb1-450c-8664-bf8ce7bd94ac admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.634750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.635665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.636001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.636339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.637236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:01.638254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:01.638254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:29:01.638254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:29:01.639128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:01.651840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 749/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:29:01.656241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.666369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce6449-052f-44ba-8788-a32f61ad4e2d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.672093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.672791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.673186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.673466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.674210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:01.675193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:01.675193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:29:01.675193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:29:01.676137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5665-3b73-47e6-92e8-ace5a496931d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:01.691057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 751/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:29:01.695421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.706310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1588e-713b-471b-9e74-273616073898 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.711378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.712278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.712558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.712863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] PATH_INFO: `/v3/projects/c333aa69c7784b098fb3befa40831345/users/b66208a17d52400b95342a24f83eeb5e/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.728940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c333aa69c7784b098fb3befa40831345, user_id=b66208a17d52400b95342a24f83eeb5e, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:01.730057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:01.730057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:29:01.730057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:29:01.732151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef5739-c010-46cb-8e82-9b7442a5303b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:01.751445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 750/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:29:01 2022] PUT /identity/v3/projects/c333aa69c7784b098fb3befa40831345/users/b66208a17d52400b95342a24f83eeb5e/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:29:01.767937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567e377-82a6-4888-b7c3-40878669183d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.768224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567e377-82a6-4888-b7c3-40878669183d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.768494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567e377-82a6-4888-b7c3-40878669183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.796365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-c567e377-82a6-4888-b7c3-40878669183d None None] MFA Rules not processed for user `b66208a17d52400b95342a24f83eeb5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:29:01.797660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c567e377-82a6-4888-b7c3-40878669183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:01.874326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 752/1502] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:29:01.888286 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf6bde-f0d0-435d-a22d-9df24cdc2391 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.888654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf6bde-f0d0-435d-a22d-9df24cdc2391 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.888926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf6bde-f0d0-435d-a22d-9df24cdc2391 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.890301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 751/1503] 10.222.0.57 () {54 vars in 963 bytes} [Thu Jun 30 12:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:29:01.895202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3aad-eba3-494c-a239-1f2a792b4de5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.895495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3aad-eba3-494c-a239-1f2a792b4de5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.895769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3aad-eba3-494c-a239-1f2a792b4de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:01.915367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-70fb3aad-eba3-494c-a239-1f2a792b4de5 None None] MFA Rules not processed for user `d61b8631879a461db96399a9b91682c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:29:01.916605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-70fb3aad-eba3-494c-a239-1f2a792b4de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:01.964801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:29:01.974243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 753/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Thu Jun 30 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:29:01.982121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf6bde-f0d0-435d-a22d-9df24cdc2391 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:01.994169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf6bde-f0d0-435d-a22d-9df24cdc2391 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:01.998562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:01.999299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:01.999588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:01.999866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:02.008619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:02.010110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:02.019476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 752/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:29:02.067993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c370b2-b3c3-47d9-afba-474867aaf8fd None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:02.068286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c370b2-b3c3-47d9-afba-474867aaf8fd None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:02.068567 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c370b2-b3c3-47d9-afba-474867aaf8fd None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:02.069924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 754/1506] 10.222.0.57 () {52 vars in 829 bytes} [Thu Jun 30 12:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:29:02.073488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:02.144764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19c17-fa48-4497-89ff-5283fcc683de service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:02.152966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:02.153854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:02.154140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:02.154382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] PATH_INFO: `/v3/users/b66208a17d52400b95342a24f83eeb5e` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:02.155722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66208a17d52400b95342a24f83eeb5e)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:02.156808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:02.156808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:29:02.156808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:29:02.158017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:02.225748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66208a17d52400b95342a24f83eeb5e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.226344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66208a17d52400b95342a24f83eeb5e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.246661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66208a17d52400b95342a24f83eeb5e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.247670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66208a17d52400b95342a24f83eeb5e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.260076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66208a17d52400b95342a24f83eeb5e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.269625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:29:02.284027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 753/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:29:02 2022] DELETE /identity/v3/users/b66208a17d52400b95342a24f83eeb5e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:29:02.288779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:02.289073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:02.289352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:02.309986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:29:02.311774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:02.377944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 755/1508] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:29:02.388329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:02.390117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:02.452366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97386990-b3ed-4ab3-9451-eaa1318b1cb1 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:02.457009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2123832-fb65-48f6-bd39-cd7229beca36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:02.457658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123832-fb65-48f6-bd39-cd7229beca36 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:02.457940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123832-fb65-48f6-bd39-cd7229beca36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:02.458213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123832-fb65-48f6-bd39-cd7229beca36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:02.470848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2123832-fb65-48f6-bd39-cd7229beca36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:02.473283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2123832-fb65-48f6-bd39-cd7229beca36 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:02.486869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 754/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:29:02.843284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:29:02.844832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:29:02.856774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34ff2d-627e-4b08-8451-52bf49560d05 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:29:02.861515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:29:02.862419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:29:02.862631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:29:02.862910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] PATH_INFO: `/v3/projects/c333aa69c7784b098fb3befa40831345` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:29:02.864197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c333aa69c7784b098fb3befa40831345)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:29:02.865251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:29:02.865251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:29:02.865251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:29:02.866441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:29:02.929522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c333aa69c7784b098fb3befa40831345', 'request_id': 'req-ea8809e8-18a7-428d-8e32-a87f3164cfce'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.941184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c333aa69c7784b098fb3befa40831345', 'request_id': 'req-ea8809e8-18a7-428d-8e32-a87f3164cfce'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.948013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c333aa69c7784b098fb3befa40831345', 'request_id': 'req-ea8809e8-18a7-428d-8e32-a87f3164cfce'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.968487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] The token cache is being invalidate because project c333aa69c7784b098fb3befa40831345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:29:02.968811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.969643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea8809e8-18a7-428d-8e32-a87f3164cfce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:29:02.971341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 756/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:29:02 2022] DELETE /identity/v3/projects/c333aa69c7784b098fb3befa40831345 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:29:12.482186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=29 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:29:12.964573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:30:47.343383 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d4cde-1695-4df5-a5f8-24c3e1d570b4 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:47.344230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d4cde-1695-4df5-a5f8-24c3e1d570b4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:47.344230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d4cde-1695-4df5-a5f8-24c3e1d570b4 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:47.345555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:30:47.350829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334f932-314e-4802-b6cb-df2c7c7c78bc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:47.351290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334f932-314e-4802-b6cb-df2c7c7c78bc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:47.351392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334f932-314e-4802-b6cb-df2c7c7c78bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:47.371692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8334f932-314e-4802-b6cb-df2c7c7c78bc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:30:47.372942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8334f932-314e-4802-b6cb-df2c7c7c78bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:30:47.434139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:30:47.592816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d4cde-1695-4df5-a5f8-24c3e1d570b4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:30:47.595180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-096d4cde-1695-4df5-a5f8-24c3e1d570b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:30:47.682735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d4cde-1695-4df5-a5f8-24c3e1d570b4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:30:47.692698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ad91d-a738-4bbe-b21b-9284cd0bb44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:30:47.693657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad91d-a738-4bbe-b21b-9284cd0bb44b service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:47.694043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad91d-a738-4bbe-b21b-9284cd0bb44b service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:47.694393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad91d-a738-4bbe-b21b-9284cd0bb44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:47.706579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ad91d-a738-4bbe-b21b-9284cd0bb44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:30:47.708198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ad91d-a738-4bbe-b21b-9284cd0bb44b service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:30:47.718067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 756/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:30:48.552919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ac13b-0894-4322-9603-7a6add1d5818 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:48.553813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ac13b-0894-4322-9603-7a6add1d5818 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:48.553813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ac13b-0894-4322-9603-7a6add1d5818 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:48.555236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:30:48.561815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac1e09-eba1-4c8b-bc91-c93e26d511da None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:48.562329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac1e09-eba1-4c8b-bc91-c93e26d511da None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:48.562823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac1e09-eba1-4c8b-bc91-c93e26d511da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:48.585182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9fac1e09-eba1-4c8b-bc91-c93e26d511da None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:30:48.586894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9fac1e09-eba1-4c8b-bc91-c93e26d511da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:30:48.598105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:30:48.754180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ac13b-0894-4322-9603-7a6add1d5818 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:30:48.770392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ac13b-0894-4322-9603-7a6add1d5818 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:30:48.771251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5289cc-1216-4733-a009-f7f042650167 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:48.771539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5289cc-1216-4733-a009-f7f042650167 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:48.771814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5289cc-1216-4733-a009-f7f042650167 None None] PATH_INFO: `/v3` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:48.773224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 758/1516] 10.222.0.13 () {54 vars in 925 bytes} [Thu Jun 30 12:30:48 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:30:48.776086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97634d9b-0235-4c83-b683-4113c6bc82fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:30:48.776763 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634d9b-0235-4c83-b683-4113c6bc82fe service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:48.777043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634d9b-0235-4c83-b683-4113c6bc82fe service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:48.777309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634d9b-0235-4c83-b683-4113c6bc82fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:48.781187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddd065-afc0-42e5-b90a-ce139b2aba6c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:48.781581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddd065-afc0-42e5-b90a-ce139b2aba6c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:48.781863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddd065-afc0-42e5-b90a-ce139b2aba6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:48.786511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97634d9b-0235-4c83-b683-4113c6bc82fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:30:48.788103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97634d9b-0235-4c83-b683-4113c6bc82fe service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:30:48.802530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 759/1517] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:30:48.805014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-afddd065-afc0-42e5-b90a-ce139b2aba6c None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:30:48.806235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-afddd065-afc0-42e5-b90a-ce139b2aba6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:30:48.874166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 759/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 30 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:30:57.548616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcc577-8870-458e-b62a-7b9955547575 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:57.549249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcc577-8870-458e-b62a-7b9955547575 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:57.549249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcc577-8870-458e-b62a-7b9955547575 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:57.550631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 760/1519] 10.222.0.57 () {54 vars in 960 bytes} [Thu Jun 30 12:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:30:57.557317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6986d-2d20-41a8-97fe-104ee04bd275 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:57.558040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6986d-2d20-41a8-97fe-104ee04bd275 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:57.558220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6986d-2d20-41a8-97fe-104ee04bd275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:57.581542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-65c6986d-2d20-41a8-97fe-104ee04bd275 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:30:57.582764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-65c6986d-2d20-41a8-97fe-104ee04bd275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:30:57.592824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 760/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Thu Jun 30 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:30:57.601241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcc577-8870-458e-b62a-7b9955547575 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:30:57.623612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcc577-8870-458e-b62a-7b9955547575 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:30:57.631798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dac121-f088-409d-8e6e-47a38666c2ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:30:57.632437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dac121-f088-409d-8e6e-47a38666c2ee service placement] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:30:57.632728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dac121-f088-409d-8e6e-47a38666c2ee service placement] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:30:57.633005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dac121-f088-409d-8e6e-47a38666c2ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:30:57.641855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dac121-f088-409d-8e6e-47a38666c2ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:30:57.643515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dac121-f088-409d-8e6e-47a38666c2ee service placement] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:30:57.653201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 761/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:31:00.064121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4a9e4-2cec-4447-b7c4-68383c11aa29 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:00.064738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4a9e4-2cec-4447-b7c4-68383c11aa29 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:00.064738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4a9e4-2cec-4447-b7c4-68383c11aa29 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:00.066081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:31:00.071161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ca1e6-a259-4193-a799-17e92a19d1bc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:00.071633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ca1e6-a259-4193-a799-17e92a19d1bc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:00.071738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ca1e6-a259-4193-a799-17e92a19d1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:00.091026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-921ca1e6-a259-4193-a799-17e92a19d1bc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:31:00.092231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-921ca1e6-a259-4193-a799-17e92a19d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:00.101137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:31:00.260390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4a9e4-2cec-4447-b7c4-68383c11aa29 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:31:00.280739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4a9e4-2cec-4447-b7c4-68383c11aa29 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:31:00.287457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4fcc9-a002-4509-8f6e-71bd8eab1b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:31:00.288092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4fcc9-a002-4509-8f6e-71bd8eab1b69 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:00.288380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4fcc9-a002-4509-8f6e-71bd8eab1b69 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:00.288656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4fcc9-a002-4509-8f6e-71bd8eab1b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:00.297019 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4fcc9-a002-4509-8f6e-71bd8eab1b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:31:00.298594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4fcc9-a002-4509-8f6e-71bd8eab1b69 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:31:00.309589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 762/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:31:01.132209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70ebd8-dd27-42db-940e-df851ff7dd09 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:01.132844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70ebd8-dd27-42db-940e-df851ff7dd09 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:01.133227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70ebd8-dd27-42db-940e-df851ff7dd09 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:01.135767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:31:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:31:01.143130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10875992-f08b-4595-bde6-b7ebfd360d9a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:01.143600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10875992-f08b-4595-bde6-b7ebfd360d9a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:01.143705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10875992-f08b-4595-bde6-b7ebfd360d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:01.163863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-10875992-f08b-4595-bde6-b7ebfd360d9a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:31:01.165116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-10875992-f08b-4595-bde6-b7ebfd360d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:01.174142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:31:01.331062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70ebd8-dd27-42db-940e-df851ff7dd09 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:31:01.347903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70ebd8-dd27-42db-940e-df851ff7dd09 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:31:01.354254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3daaca-7252-436d-af8e-7e25b9cddaa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:31:01.354903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3daaca-7252-436d-af8e-7e25b9cddaa2 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:01.355190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3daaca-7252-436d-af8e-7e25b9cddaa2 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:01.355463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3daaca-7252-436d-af8e-7e25b9cddaa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:01.363993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3daaca-7252-436d-af8e-7e25b9cddaa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:31:01.365662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3daaca-7252-436d-af8e-7e25b9cddaa2 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:31:01.375613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 764/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:31:05.469327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc3f1-a01a-4b16-87dd-20638c4c0672 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:05.470000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc3f1-a01a-4b16-87dd-20638c4c0672 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:05.470000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc3f1-a01a-4b16-87dd-20638c4c0672 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:05.471345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:31:05.477162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef125f1a-2f70-4662-a904-17ba6d9c0a57 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:05.477668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef125f1a-2f70-4662-a904-17ba6d9c0a57 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:05.477668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef125f1a-2f70-4662-a904-17ba6d9c0a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:05.496675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ef125f1a-2f70-4662-a904-17ba6d9c0a57 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:31:05.497853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ef125f1a-2f70-4662-a904-17ba6d9c0a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:05.506931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:31:05.543587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc3f1-a01a-4b16-87dd-20638c4c0672 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:31:05.546082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cb4dc3f1-a01a-4b16-87dd-20638c4c0672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:05.608062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:31:05.615739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc3f1-a01a-4b16-87dd-20638c4c0672 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:31:05.619663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a01fea-067b-44d7-9266-d9a4a0890108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:31:05.620319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a01fea-067b-44d7-9266-d9a4a0890108 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:05.620603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a01fea-067b-44d7-9266-d9a4a0890108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:05.620909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a01fea-067b-44d7-9266-d9a4a0890108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:05.630388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a01fea-067b-44d7-9266-d9a4a0890108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:31:05.631915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a01fea-067b-44d7-9266-d9a4a0890108 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:31:05.641381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 765/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:31:07.685430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8d54-eb95-4d0b-8fb6-df34daec3414 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:07.686346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8d54-eb95-4d0b-8fb6-df34daec3414 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:07.686346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8d54-eb95-4d0b-8fb6-df34daec3414 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:07.688581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:31:07.696499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368ed6f-f436-4c9f-a3a4-f57edc34bfff None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:07.696990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368ed6f-f436-4c9f-a3a4-f57edc34bfff None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:07.697092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368ed6f-f436-4c9f-a3a4-f57edc34bfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:07.717008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c368ed6f-f436-4c9f-a3a4-f57edc34bfff None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:31:07.718208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c368ed6f-f436-4c9f-a3a4-f57edc34bfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:07.726808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:31:07.763054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8d54-eb95-4d0b-8fb6-df34daec3414 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:31:07.775549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8d54-eb95-4d0b-8fb6-df34daec3414 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:31:07.779670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf602d1e-320b-441a-ab06-f93f5bd6ebbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:31:07.780289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf602d1e-320b-441a-ab06-f93f5bd6ebbb service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:07.780573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf602d1e-320b-441a-ab06-f93f5bd6ebbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:07.780881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf602d1e-320b-441a-ab06-f93f5bd6ebbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:07.790705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf602d1e-320b-441a-ab06-f93f5bd6ebbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:31:07.792132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf602d1e-320b-441a-ab06-f93f5bd6ebbb service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:31:07.803982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 767/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:31:11.397073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68294162-f6fe-49f4-80aa-1069a00e7b46 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:11.397738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68294162-f6fe-49f4-80aa-1069a00e7b46 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:11.397738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68294162-f6fe-49f4-80aa-1069a00e7b46 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:11.399246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:31:11.405227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305326eb-84fc-4e39-a558-44b6de7fdb32 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:11.405745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305326eb-84fc-4e39-a558-44b6de7fdb32 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:11.405877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305326eb-84fc-4e39-a558-44b6de7fdb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:11.426787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-305326eb-84fc-4e39-a558-44b6de7fdb32 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:31:11.428265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-305326eb-84fc-4e39-a558-44b6de7fdb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:11.437142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:31:11.739476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68294162-f6fe-49f4-80aa-1069a00e7b46 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:31:11.741620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-68294162-f6fe-49f4-80aa-1069a00e7b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:11.804215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68294162-f6fe-49f4-80aa-1069a00e7b46 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:31:11.808215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34969e41-29f2-4a7d-8be4-c2d35580122d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:31:11.808893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34969e41-29f2-4a7d-8be4-c2d35580122d service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:11.809174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34969e41-29f2-4a7d-8be4-c2d35580122d service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:11.809446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34969e41-29f2-4a7d-8be4-c2d35580122d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:11.819757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34969e41-29f2-4a7d-8be4-c2d35580122d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:31:11.821295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34969e41-29f2-4a7d-8be4-c2d35580122d service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:31:11.831305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 768/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:31:13.907364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d6f74-0bbb-4278-81d9-f1c0bc046283 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:13.907997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d6f74-0bbb-4278-81d9-f1c0bc046283 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:13.907997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d6f74-0bbb-4278-81d9-f1c0bc046283 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:13.909356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:31:13.915936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7e463-209a-432f-873d-9556e9f6aede None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:13.916484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7e463-209a-432f-873d-9556e9f6aede None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:13.916616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7e463-209a-432f-873d-9556e9f6aede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:13.945459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7ad7e463-209a-432f-873d-9556e9f6aede None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:31:13.946688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7ad7e463-209a-432f-873d-9556e9f6aede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:13.956211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:31:14.275714 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d6f74-0bbb-4278-81d9-f1c0bc046283 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:31:14.277918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e74d6f74-0bbb-4278-81d9-f1c0bc046283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:31:14.286956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=61 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:31:14.296023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d6f74-0bbb-4278-81d9-f1c0bc046283 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:31:14.301210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edb4e8-e32f-4b27-8f6f-7d84aff673ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:31:14.301878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edb4e8-e32f-4b27-8f6f-7d84aff673ff service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:31:14.302157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edb4e8-e32f-4b27-8f6f-7d84aff673ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:31:14.302827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edb4e8-e32f-4b27-8f6f-7d84aff673ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:31:14.313179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edb4e8-e32f-4b27-8f6f-7d84aff673ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:31:14.314630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edb4e8-e32f-4b27-8f6f-7d84aff673ff service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:31:14.324801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 770/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:31:23.937089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=36 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:31:24.320430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:44.931058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:44.931740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:44.931740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:44.959414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:44.960659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:44.973250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 770/1540] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:44.982341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:44.982934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:44.982934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.011667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.012884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.024757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 771/1541] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.030219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.041084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed790e-2e57-4283-8a47-9ef49774a067 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.052031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.052763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.053040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.053309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.054044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.054400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.055432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:32:45.055432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:32:45.055432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:32:45.056324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09d809-2b71-441c-8685-0eff33d012b5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.075787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 771/1542] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.084318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.094826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ae421-278d-429f-a6d1-800b37ea3aec None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.103688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.104377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.104679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.104959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.105675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.106708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:32:45.106708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:32:45.106708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:32:45.107936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c018c-3354-458d-8659-cb3b36f8a0c2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.121866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.122203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.122420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.151722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 772/1543] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:45.153103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e34964-ec7e-496c-9c72-400caa4e7793 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.153392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e34964-ec7e-496c-9c72-400caa4e7793 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.153624 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.153744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e34964-ec7e-496c-9c72-400caa4e7793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.154879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.180701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a3e34964-ec7e-496c-9c72-400caa4e7793 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.184484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a3e34964-ec7e-496c-9c72-400caa4e7793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.262345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 772/1544] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:32:45.263448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.284456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b88cfc-732c-4b49-afbf-5f3bf88c0ffb None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.294173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.294870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.295152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.295420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.296503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.297581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:32:45.297581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:32:45.297581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:32:45.298796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10589a6-a74c-4f07-b15f-9605e3166362 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.300052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 773/1545] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.301913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0b246-d58b-433f-b75b-108e70f899a3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.302193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0b246-d58b-433f-b75b-108e70f899a3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.302466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0b246-d58b-433f-b75b-108e70f899a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.322759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-27d0b246-d58b-433f-b75b-108e70f899a3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.326905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 773/1546] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/users => generated 288 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:45.327060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-27d0b246-d58b-433f-b75b-108e70f899a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.327157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33aca-1cec-437c-97cb-d9bb4a5f7c46 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.327157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33aca-1cec-437c-97cb-d9bb4a5f7c46 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.327157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33aca-1cec-437c-97cb-d9bb4a5f7c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.334565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 774/1547] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.335832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4fed-4121-4394-8c35-85bba86d554e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.336126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4fed-4121-4394-8c35-85bba86d554e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.336404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4fed-4121-4394-8c35-85bba86d554e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.352940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-08b33aca-1cec-437c-97cb-d9bb4a5f7c46 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.354289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-08b33aca-1cec-437c-97cb-d9bb4a5f7c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.358110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6e4d4fed-4121-4394-8c35-85bba86d554e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.359340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6e4d4fed-4121-4394-8c35-85bba86d554e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.362980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 774/1548] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.364208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739f692-1cbe-4887-af32-8a54da443fac None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.364497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739f692-1cbe-4887-af32-8a54da443fac None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.364810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739f692-1cbe-4887-af32-8a54da443fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.368336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 775/1549] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.373456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.373962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.374238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.384890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b739f692-1cbe-4887-af32-8a54da443fac None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.390310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b739f692-1cbe-4887-af32-8a54da443fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.396840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 775/1550] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.397495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.398106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e1a85-ea09-47e7-afe7-4f4f63a0cf57 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.398394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e1a85-ea09-47e7-afe7-4f4f63a0cf57 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.398667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e1a85-ea09-47e7-afe7-4f4f63a0cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.398780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.407806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 776/1551] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.408859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.417490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-715e1a85-ea09-47e7-afe7-4f4f63a0cf57 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.418704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-715e1a85-ea09-47e7-afe7-4f4f63a0cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.418862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd523125-0059-4b79-9c94-34bc551ef0b7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.424860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e7348-7ba2-4320-a0a8-9d2c880717b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.425528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7348-7ba2-4320-a0a8-9d2c880717b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.425807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7348-7ba2-4320-a0a8-9d2c880717b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.426076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7348-7ba2-4320-a0a8-9d2c880717b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.426801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e7348-7ba2-4320-a0a8-9d2c880717b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.427058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 776/1552] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.428294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.428410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e7348-7ba2-4320-a0a8-9d2c880717b9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.428579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.428886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.439692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 777/1553] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:45.440994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebea1b-5f72-4b59-9a43-9c9e928b7682 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.441278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebea1b-5f72-4b59-9a43-9c9e928b7682 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.441550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebea1b-5f72-4b59-9a43-9c9e928b7682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.449405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.450620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.458921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 777/1554] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.459670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1cebea1b-5f72-4b59-9a43-9c9e928b7682 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.459811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.460867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1cebea1b-5f72-4b59-9a43-9c9e928b7682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.469572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 778/1555] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.469811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69da5-c186-4321-bd16-f73a58a63ef6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.470929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4aadf-d310-453a-b08a-2ee321aedb3e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.471224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4aadf-d310-453a-b08a-2ee321aedb3e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.471656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4aadf-d310-453a-b08a-2ee321aedb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.474652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.475315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.475597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.475926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.476885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.477198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.478821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda841f-6190-4ce4-afd5-194424e1e2a1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.490553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f6d4aadf-d310-453a-b08a-2ee321aedb3e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.490716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 778/1556] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.491778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f6d4aadf-d310-453a-b08a-2ee321aedb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.491972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fe60b-475c-4841-b507-10982b27f633 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.492238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fe60b-475c-4841-b507-10982b27f633 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.492518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fe60b-475c-4841-b507-10982b27f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.504259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 779/1557] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.505823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.506110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.506416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.514147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-125fe60b-475c-4841-b507-10982b27f633 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.515591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-125fe60b-475c-4841-b507-10982b27f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.523742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 779/1558] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.524364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fe60b-475c-4841-b507-10982b27f633 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.525597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.527033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.534035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fe60b-475c-4841-b507-10982b27f633 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.535212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 780/1559] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.535989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.542561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.543225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.543509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.543782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.544511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.544851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.545325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9ec10-a356-4fd6-8416-2c87d4a774ff None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.546628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.550328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bcfbc-531b-4177-8d34-7ff76a6e3475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.550974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bcfbc-531b-4177-8d34-7ff76a6e3475 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.551254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bcfbc-531b-4177-8d34-7ff76a6e3475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.551526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bcfbc-531b-4177-8d34-7ff76a6e3475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.552274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bcfbc-531b-4177-8d34-7ff76a6e3475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.553921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bcfbc-531b-4177-8d34-7ff76a6e3475 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.558745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 780/1560] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.559532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.567687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 781/1561] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:45.568963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e30f21-ffaf-4c67-ac9a-22a543090dba admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.569093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.569274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.569515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.574474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.575119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.575399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.575672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.576388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.576757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.579360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.587190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:45.588419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:45.591595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 781/1562] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.592357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.599713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 782/1563] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:45.599713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.603881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beeb58-711e-4588-a1c6-498647026646 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.606369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc38ce-caa2-43a1-926a-df4efbaeec43 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.608731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.609390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.609677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.609953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.610641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.611422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.612070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.612350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.612542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.612678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.613359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.613665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.615247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.627859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 783/1564] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.628831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.641018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745547fe-ba86-4f16-855c-39a3d403aec7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.648110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 782/1565] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:45.648110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.649004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.649704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.649986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.650259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.650985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.651291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.652927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.665030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 784/1566] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.665878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.678011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9c83d-08f7-4dc2-a9ad-dc91e5ac5fdc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.685345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:45.717428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:45.719926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0b64f-a480-43d0-96c1-2b4cb0685580 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.726068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.726766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.727048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.727327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.728076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.728531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.730245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.743429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 783/1567] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.744275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.744490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.745219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.745499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.745771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.746457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.748435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.755104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cf49b-84ee-4093-9309-bde4871314e0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.760565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.761648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.761940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.762219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] PATH_INFO: `/v3/projects/b60d10dba1d947cc97149a063a810129/users/6ddb38b4d6864a6c96b84c9a4092eea8/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.780186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 785/1568] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:45.782169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60d10dba1d947cc97149a063a810129, user_id=6ddb38b4d6864a6c96b84c9a4092eea8, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.784409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.785803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.802275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 784/1569] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:45 2022] PUT /identity/v3/projects/b60d10dba1d947cc97149a063a810129/users/6ddb38b4d6864a6c96b84c9a4092eea8/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:45.803487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.852967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6543-7964-4ca4-bbd7-500b9cd28d53 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.869554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f2df-96f8-470d-bfb3-5c3edd9978ce admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.875298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.875967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.876250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.876524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.877668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:45.878002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.880000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.892275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 785/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:45.897838 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.908292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae64a-d4d0-4f2f-a4a6-8f7ea956a7eb admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:45.908805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.909489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.909831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.910036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.910753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.912711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.913112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:45.913830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:45.914045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:45.914322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:45.915041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:45.916965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:45.938996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 786/1571] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:45.939799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.952397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 786/1572] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:45.952397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:45.998953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:46.004250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc953-e977-4235-a392-2a3c39254fed admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.016916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a1850-7149-460b-9bf2-16a859993605 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.021712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.022390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.022674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.022949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.023674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.025656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.040757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:46.055228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 787/1573] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.056207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.066337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.067024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.067330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.067622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.068314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.070299 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.095575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 787/1574] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:45 2022] POST /identity/v3/users => generated 302 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.098176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.108858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c0f91-6c23-44ad-9137-22e8256a4603 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.129564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb169eb-f493-44d0-b1f4-196542dbfaf3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.134372 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a05634-ec81-4129-928e-39727510b336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.135098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a05634-ec81-4129-928e-39727510b336 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.135380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a05634-ec81-4129-928e-39727510b336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.135652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a05634-ec81-4129-928e-39727510b336 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.136468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a05634-ec81-4129-928e-39727510b336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.138460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a05634-ec81-4129-928e-39727510b336 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.168729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 788/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.168729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.168729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.168729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.177347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.178048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.178328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.178628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.179362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.181359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.185680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:46.187137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:46.204598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 788/1576] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.205783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.215775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065a837-7535-409f-bde1-14c39e032afd admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.255505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 789/1577] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:46.256341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.268362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b33396-12cc-4d6f-882f-6f3e51f69a51 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.274866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.275550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.275821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.276088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.276896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.278881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.281628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.282386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.282740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.283082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.283840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.285890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.306145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 790/1578] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.306145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.314926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee855c3-f6fb-40b4-8ec4-b8721ee70ec9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.318821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 789/1579] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.319756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.322742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.323513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.323798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.324074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.324808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.326741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.329621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:46.347933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 791/1580] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/users => generated 283 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.351882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.362258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab4059-5281-43cc-9674-c4f25912b85d admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.387243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cdc31-a113-4583-b3e2-bbd4c6a92010 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.393242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.393908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.394177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.394471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.395132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.397142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.407795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:46.418973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 790/1581] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.419899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.424000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.425112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.425400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.425903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.426776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.428399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.430747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726b22d-6713-4402-834b-53a8e209c4f2 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.436557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.437109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.437388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.437666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.438356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.441705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 792/1582] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.441705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.441930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.451705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8c203-d0e5-4852-a648-5f53dca6f5d3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.458017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.458689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.458971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.459253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.459999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.461609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.462538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 791/1583] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.463371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.475575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 793/1584] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.476565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.486758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab857f61-46ac-41ee-a1d6-5700e51cbce5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.491674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.492340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.492611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.492938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.493692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.495290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.505963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 794/1585] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.509898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.519631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f4f1f-17bd-4c87-9409-635179afa7a8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.520943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b72be-b2a6-451e-941c-d1fbf4ab5276 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.524426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.524688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.525127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.525369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.525479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.525667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.525764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.525939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.526356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.528320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.535596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.537116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.547271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 792/1586] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:46.548081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.549916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 795/1587] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:46 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:46.550619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.558050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841ea2-a2f4-4774-b04b-7c951cf76432 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.560199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5e0da-4dac-4a86-ae47-eeb9d6bda46a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.563543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.564292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.564576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.564882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.565116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.565662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.565770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.566056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.566329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.567049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.567340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.568824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.579367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 796/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.580422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.581328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 793/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.585518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.592238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0027fb-e83e-4ee7-b05c-783c1d268965 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.596795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fead1-e5d0-42f9-9c47-f50dcd7b370e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.597197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.597878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.598161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.598442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.599314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.601040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.602185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.602868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.603152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.603431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.604163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.605836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.617385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 797/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.617385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.620096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 794/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.621112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.628956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e50f-57bf-4661-a4ad-5e4602e4143f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.632678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6f3a5-d3cb-4135-8b3a-bcdf1e2396b6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.634132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.634836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.635123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.635542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.636255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.637949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.638173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.639028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.639462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.639793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.640537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.642189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.650414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 798/1592] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.651984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.655388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 795/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.659640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.663698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd5a03-4c1a-4cd3-9a19-1d8de2504bd6 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.669805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.669993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ff9d-52fd-46dc-8abd-22a55a738f1d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.670922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.671526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.672001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.672954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.675126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.675400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.676079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.676464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.676652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.677391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.678980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.687787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 799/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.688477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.690149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 796/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.691058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.701958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a2a-58e9-4eed-a575-c43d2abc2ae1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.703375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c71aa8-0308-49bf-a2c4-49f90aa73324 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.706928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.707835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.708113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.708383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] PATH_INFO: `/v3/projects/f6f6e3f2d7c946a28515b911281a4d34/users/da4fcf7be39e4b01a4c93e91b7cde4ba/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.708780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.709499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.709796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.710077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.710935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.712780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.724685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f6e3f2d7c946a28515b911281a4d34, user_id=da4fcf7be39e4b01a4c93e91b7cde4ba, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.725741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:32:46.725741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:32:46.725741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:32:46.727657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.728831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 800/1596] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.730543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.742012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cb5b-a994-4a1d-8627-79e4c1387082 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.744459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 797/1597] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:46 2022] PUT /identity/v3/projects/f6f6e3f2d7c946a28515b911281a4d34/users/da4fcf7be39e4b01a4c93e91b7cde4ba/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:46.745318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.761476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:46.813763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.814446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.814729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.815295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.815737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.817512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.822897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7142-92a5-4e3f-a12a-79ac2cbd89a1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.828564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 801/1598] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:46.829374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.832381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.833351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.833762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.833947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] PATH_INFO: `/v3/projects/e39c3bfdb43441b1a6241f5d926b62be/users/04b8b5e97ba6419c999201b8699d0c47/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.840457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a969b44-fc69-4ddb-ae53-92e631e0e1ea admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.846221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.847182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.847462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.847736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] PATH_INFO: `/v3/projects/8fe4f134e4084a878a5e5cf59d96630f/users/3b688425466947198520ec7554bb49f7/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:46.850219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39c3bfdb43441b1a6241f5d926b62be, user_id=04b8b5e97ba6419c999201b8699d0c47, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.853446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.863566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe4f134e4084a878a5e5cf59d96630f, user_id=3b688425466947198520ec7554bb49f7, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:46.866456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:46.868592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 798/1599] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:46 2022] PUT /identity/v3/projects/e39c3bfdb43441b1a6241f5d926b62be/users/04b8b5e97ba6419c999201b8699d0c47/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:46.869615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.889134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 802/1600] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:46 2022] PUT /identity/v3/projects/8fe4f134e4084a878a5e5cf59d96630f/users/3b688425466947198520ec7554bb49f7/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:46.890525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:46.919155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:46.961528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9789b-5b7a-4d35-84b3-bce4e28398f9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.976889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353bc49-65d3-42fc-a995-8f6e582668f0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:46.984837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:46.986022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:46.986408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:46.986785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] PATH_INFO: `/v3/projects/23c307d3f958455f81fd31898ab893ff/users/43d318f3788341c2b03adff2a77c0cd0/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.004873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c307d3f958455f81fd31898ab893ff, user_id=43d318f3788341c2b03adff2a77c0cd0, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.007797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.025453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5852e-be92-40af-acd6-a99972a774f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.026076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5852e-be92-40af-acd6-a99972a774f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.026171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5852e-be92-40af-acd6-a99972a774f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.026573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5852e-be92-40af-acd6-a99972a774f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.027532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5852e-be92-40af-acd6-a99972a774f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.028846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 803/1601] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:46 2022] PUT /identity/v3/projects/23c307d3f958455f81fd31898ab893ff/users/43d318f3788341c2b03adff2a77c0cd0/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:47.029344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5852e-be92-40af-acd6-a99972a774f1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.031601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.041241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 799/1602] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:47.042533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.042821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.043097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.074598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] MFA Rules not processed for user `da4fcf7be39e4b01a4c93e91b7cde4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.076146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.107728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:47.113560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b06786-8d2d-4851-a3fd-22b302a51875 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.118154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d06fea-a353-480e-bf72-c63e4625a216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.119137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d06fea-a353-480e-bf72-c63e4625a216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.119361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d06fea-a353-480e-bf72-c63e4625a216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.119695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d06fea-a353-480e-bf72-c63e4625a216 admin admin] PATH_INFO: `/v3/projects/162bbd0a5f74414e935a585d0ad5c09b/users/6dea5da5d96f435d8e8b0a572e3b2313/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.136957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d06fea-a353-480e-bf72-c63e4625a216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162bbd0a5f74414e935a585d0ad5c09b, user_id=6dea5da5d96f435d8e8b0a572e3b2313, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.140349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d06fea-a353-480e-bf72-c63e4625a216 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.152034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 800/1603] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.152034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.162204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 804/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:47 2022] PUT /identity/v3/projects/162bbd0a5f74414e935a585d0ad5c09b/users/6dea5da5d96f435d8e8b0a572e3b2313/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:47.163808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95234-4760-43e2-a80e-f784969c2b82 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.164160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95234-4760-43e2-a80e-f784969c2b82 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.164533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95234-4760-43e2-a80e-f784969c2b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.173769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd85d2-3249-45fb-a77e-1be4d746b291 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.198693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-eba95234-4760-43e2-a80e-f784969c2b82 None None] MFA Rules not processed for user `04b8b5e97ba6419c999201b8699d0c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.200058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-eba95234-4760-43e2-a80e-f784969c2b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.247738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d1ab6-fc83-4f36-8f74-2d102db76723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.248862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1ab6-fc83-4f36-8f74-2d102db76723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.249368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1ab6-fc83-4f36-8f74-2d102db76723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.249700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1ab6-fc83-4f36-8f74-2d102db76723 admin admin] PATH_INFO: `/v3/projects/663ec7a59bc147299f904d93d119c5e7/users/26807e5e7aab425eb28e781289794265/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.275312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d1ab6-fc83-4f36-8f74-2d102db76723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663ec7a59bc147299f904d93d119c5e7, user_id=26807e5e7aab425eb28e781289794265, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.278708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d1ab6-fc83-4f36-8f74-2d102db76723 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.288803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 805/1605] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.291048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.291368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.291701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.303162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 801/1606] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:47 2022] PUT /identity/v3/projects/663ec7a59bc147299f904d93d119c5e7/users/26807e5e7aab425eb28e781289794265/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:47.310349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.310994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.311707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.344515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.346453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.353028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] MFA Rules not processed for user `43d318f3788341c2b03adff2a77c0cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.355033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.358280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:47.451220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 802/1607] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.452039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 806/1608] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.452734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.453747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.467082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0552615-7753-43e7-b8d9-cec377a91e2e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.473198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d73ad8-26b6-4f2b-8c56-712fa4a723d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.474426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73ad8-26b6-4f2b-8c56-712fa4a723d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.474961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73ad8-26b6-4f2b-8c56-712fa4a723d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.475415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73ad8-26b6-4f2b-8c56-712fa4a723d2 admin admin] PATH_INFO: `/v3/projects/c7be5e06db6743c18915c8b326f275a9/users/530fc1cea18b41818f95ef5462e9682f/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.489959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc6627-4444-4857-a27a-93a64d972696 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.498991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d73ad8-26b6-4f2b-8c56-712fa4a723d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7be5e06db6743c18915c8b326f275a9, user_id=530fc1cea18b41818f95ef5462e9682f, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.501978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d73ad8-26b6-4f2b-8c56-712fa4a723d2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.503309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dde95-2daa-4557-907c-d62921b29d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.505886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dde95-2daa-4557-907c-d62921b29d08 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.507170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dde95-2daa-4557-907c-d62921b29d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.508182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dde95-2daa-4557-907c-d62921b29d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.510075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dde95-2daa-4557-907c-d62921b29d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.515429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dde95-2daa-4557-907c-d62921b29d08 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.531058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 803/1609] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:47 2022] PUT /identity/v3/projects/c7be5e06db6743c18915c8b326f275a9/users/530fc1cea18b41818f95ef5462e9682f/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:47.531058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304c8a2-6964-4c98-8a72-8418d96b7976 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.531058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304c8a2-6964-4c98-8a72-8418d96b7976 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.531058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304c8a2-6964-4c98-8a72-8418d96b7976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.553598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4304c8a2-6964-4c98-8a72-8418d96b7976 None None] MFA Rules not processed for user `6dea5da5d96f435d8e8b0a572e3b2313`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.554896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4304c8a2-6964-4c98-8a72-8418d96b7976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.561760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:47.587735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 807/1610] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/projects => generated 403 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:47.590379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.590911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.591432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.627299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.628946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.694281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 804/1611] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.697184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b81529-8bda-4736-975b-1b171ab78f11 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.697699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b81529-8bda-4736-975b-1b171ab78f11 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.698157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b81529-8bda-4736-975b-1b171ab78f11 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.699984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 805/1612] 10.222.0.57 () {54 vars in 961 bytes} [Thu Jun 30 12:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:32:47.709097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 808/1613] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.709097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.709097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.710466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b81529-8bda-4736-975b-1b171ab78f11 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.727466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b81529-8bda-4736-975b-1b171ab78f11 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.733612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43cff-b14c-4944-b61e-a4e3485050b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.734481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43cff-b14c-4944-b61e-a4e3485050b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.734770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43cff-b14c-4944-b61e-a4e3485050b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.735481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43cff-b14c-4944-b61e-a4e3485050b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.736248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf43cff-b14c-4944-b61e-a4e3485050b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.738706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf43cff-b14c-4944-b61e-a4e3485050b4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.781943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 806/1614] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:47.783518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.784065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.784363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.796652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cb778-39b5-4ed4-ae45-22b728e71833 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.825649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] MFA Rules not processed for user `530fc1cea18b41818f95ef5462e9682f`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:47.827909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:47.858146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.859001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.859113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.859551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.864208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:47.940873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 807/1615] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:47.941724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.958060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.960205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:47.972516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 809/1616] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:47.973719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:47.984059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecafa0d-245c-4a2c-9000-9a5d5987b386 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:47.989650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7115b02-6277-4801-a443-6aee08b5a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:47.991058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7115b02-6277-4801-a443-6aee08b5a833 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:47.991590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7115b02-6277-4801-a443-6aee08b5a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:47.991993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7115b02-6277-4801-a443-6aee08b5a833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:47.992825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7115b02-6277-4801-a443-6aee08b5a833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:47.995330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7115b02-6277-4801-a443-6aee08b5a833 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.019964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cb971-036a-4a6a-91df-12dd81e80b26 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.025808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.026718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.026914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.027196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.028546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.032844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.042287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 810/1617] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:48.042287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.042287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.042287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.058600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 808/1618] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:47 2022] POST /identity/v3/users => generated 308 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:48.059864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.067155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:48.070923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:48.071858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ba30f-c59c-44ad-baf1-5a08c8a51d23 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.073277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:48.155591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.156680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.157431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.157788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.188246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 811/1619] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Jun 30 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:48.189291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.312655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.314213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.325213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 809/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:48.325838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.335771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d73178-aef1-483d-bf0a-2c44e373e63c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.342597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.343828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.344896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.345873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.346782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.350945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.385087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:48.395268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 812/1621] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:48 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:48.396382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.421944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861725e9-ca45-4d0b-bde7-9958aad63267 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.429681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.431921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.431921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.431921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.433097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.435841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.439190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f135a84-5639-411b-9f01-e520a9111585 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.448240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.449302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.449757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.450221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.472166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 813/1622] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:48 2022] POST /identity/v3/users => generated 289 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:48.473650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.504329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90ceb2-d44c-43ad-bba5-41daeaff9c97 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.513213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.514129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.514315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.515887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.516761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.518905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.545312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.552724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.554438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 814/1623] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:48.555645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.621107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050f6c0-2fb2-4e4f-9236-c5713f0e1202 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.621650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 810/1624] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:48.621650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.621983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.621983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.621983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.621983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.644221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19320ed5-c4be-4edb-8c95-17df75cfce6d service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.669615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.670609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.671814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.672303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.675343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.679178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.682414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:48.698479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 811/1625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:48.712485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.732715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8486f4-1ef0-4611-b3e9-498821bcdda6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.753852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.755065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.755459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.756008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.757496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.759768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.769379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.771039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.785503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 812/1626] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:48.800578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 815/1627] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:48.801346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.807908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.824355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33143abd-9855-4088-8cd8-4d15ab075ca6 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.826425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d282a-61ce-4679-97a1-9c9f95c67408 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:48.831752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.833082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.833471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.834826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] PATH_INFO: `/v3/projects/1c29bb176b664eeca69c162be13cc1fc/users/f75572279a1f4560a63af18ad072b3c3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.838766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:48.840419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:48.840946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:48.841455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] PATH_INFO: `/v3/projects/4d53b3e2a04f413982bdaae8a77fd35c/users/16a1b526126144ccae4669d3f4489ec4/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:48.873378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d53b3e2a04f413982bdaae8a77fd35c, user_id=16a1b526126144ccae4669d3f4489ec4, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.876693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.887522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c29bb176b664eeca69c162be13cc1fc, user_id=f75572279a1f4560a63af18ad072b3c3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:48.892347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:48.898321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 816/1628] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:48 2022] PUT /identity/v3/projects/4d53b3e2a04f413982bdaae8a77fd35c/users/16a1b526126144ccae4669d3f4489ec4/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:48.899476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:48.928273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 813/1629] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:48 2022] PUT /identity/v3/projects/1c29bb176b664eeca69c162be13cc1fc/users/f75572279a1f4560a63af18ad072b3c3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:48.930712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.047170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ad1c3-278f-4881-b896-dc78322a1243 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.090714 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a4e03-7f0f-4409-b7ec-0fd84b4cabb8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.097138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.098271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.098715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.098995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.099791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.102241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.116457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:49.117888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 814/1630] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:49.118966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.133093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.134229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.134629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.135127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] PATH_INFO: `/v3/projects/250c910483fe4ec89bedf5b52ae8f1d9/users/1428a95dde0347bb82d36b959da0aee9/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.135944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a6dbe-8bcf-46e2-9b76-35bdc144de58 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.146251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.146957 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.147242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.147518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.148263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.149935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.152953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=250c910483fe4ec89bedf5b52ae8f1d9, user_id=1428a95dde0347bb82d36b959da0aee9, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.156048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.170258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 815/1631] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:49.171333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.180218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 817/1632] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:48 2022] PUT /identity/v3/projects/250c910483fe4ec89bedf5b52ae8f1d9/users/1428a95dde0347bb82d36b959da0aee9/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:49.181195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.199745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f31c56-a03d-45a4-9883-5e959261d270 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.280433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cbb7-2dbe-47f3-9c85-70afdc9ca4fa admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.283308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.284357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.284670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.285425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] PATH_INFO: `/v3/projects/1c29bb176b664eeca69c162be13cc1fc/users/f75572279a1f4560a63af18ad072b3c3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.287718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c29bb176b664eeca69c162be13cc1fc, user_id=f75572279a1f4560a63af18ad072b3c3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.312716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.339373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 818/1633] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:49 2022] PUT /identity/v3/projects/1c29bb176b664eeca69c162be13cc1fc/users/f75572279a1f4560a63af18ad072b3c3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:49.341388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.359571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772653b-061d-4b00-a898-60db3e553c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.360534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772653b-061d-4b00-a898-60db3e553c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.360846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772653b-061d-4b00-a898-60db3e553c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.361197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772653b-061d-4b00-a898-60db3e553c73 admin admin] PATH_INFO: `/v3/projects/4d53b3e2a04f413982bdaae8a77fd35c/users/16a1b526126144ccae4669d3f4489ec4/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.368785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772653b-061d-4b00-a898-60db3e553c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d53b3e2a04f413982bdaae8a77fd35c, user_id=16a1b526126144ccae4669d3f4489ec4, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.371857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772653b-061d-4b00-a898-60db3e553c73 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.414285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 816/1634] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:49 2022] PUT /identity/v3/projects/4d53b3e2a04f413982bdaae8a77fd35c/users/16a1b526126144ccae4669d3f4489ec4/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:49.416473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.417284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.417284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.482049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] MFA Rules not processed for user `f75572279a1f4560a63af18ad072b3c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:49.482049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:49.512875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536a0c8-9dcc-4b68-bc30-f0d546b04cab admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.570286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:49.605024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229832-50af-461a-8ed9-d8655f3fd7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.605865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229832-50af-461a-8ed9-d8655f3fd7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.606155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229832-50af-461a-8ed9-d8655f3fd7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.606497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229832-50af-461a-8ed9-d8655f3fd7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.607257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9229832-50af-461a-8ed9-d8655f3fd7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.609337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9229832-50af-461a-8ed9-d8655f3fd7d9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.625799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 819/1635] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:49.630440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dfb9-c93c-4310-96fa-c3cc2ada9d9e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.630440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dfb9-c93c-4310-96fa-c3cc2ada9d9e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.630440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dfb9-c93c-4310-96fa-c3cc2ada9d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.681741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:49.689887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e9b8dfb9-c93c-4310-96fa-c3cc2ada9d9e None None] MFA Rules not processed for user `16a1b526126144ccae4669d3f4489ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:49.691157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e9b8dfb9-c93c-4310-96fa-c3cc2ada9d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:49.694247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 817/1636] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:49.695233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.720289 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfe002-f64f-425a-ad07-1fe00f98996c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:49.749366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:49.750231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.750546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.750824 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] PATH_INFO: `/v3/projects/250c910483fe4ec89bedf5b52ae8f1d9/users/1428a95dde0347bb82d36b959da0aee9/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.760087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] RBAC: Authorizing `identity:create_grant(project_id=250c910483fe4ec89bedf5b52ae8f1d9, user_id=1428a95dde0347bb82d36b959da0aee9, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:49.778054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:49.817872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 818/1637] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:49 2022] PUT /identity/v3/projects/250c910483fe4ec89bedf5b52ae8f1d9/users/1428a95dde0347bb82d36b959da0aee9/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:49.829882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:49.859850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 820/1638] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:49.862020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:49.862686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:49.863138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:49.961677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] MFA Rules not processed for user `1428a95dde0347bb82d36b959da0aee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:49.962944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:50.001327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860180fd-a392-41b6-8d59-ee91aa0b856d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:50.018505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c1356-995a-4842-92ad-6d00ac778c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:50.019443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1356-995a-4842-92ad-6d00ac778c9f service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.019961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1356-995a-4842-92ad-6d00ac778c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.020798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1356-995a-4842-92ad-6d00ac778c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.129868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 821/1639] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:50.132555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:50.145990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a1739-d8f3-45de-9597-49ef3fc443bc None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:50.155952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:50.157288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.158727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.159014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.168922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c1356-995a-4842-92ad-6d00ac778c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:50.170558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c1356-995a-4842-92ad-6d00ac778c9f service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:50.193061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 819/1640] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:49 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:50.196718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fac0-f4fa-4c33-b2a2-2039856134bb None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.197015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fac0-f4fa-4c33-b2a2-2039856134bb None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.197330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fac0-f4fa-4c33-b2a2-2039856134bb None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.198733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 820/1641] 10.222.0.57 () {54 vars in 961 bytes} [Thu Jun 30 12:32:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:32:50.211646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be214e8f-032d-4c5d-8a73-855914d5d083 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.211940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be214e8f-032d-4c5d-8a73-855914d5d083 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.212223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be214e8f-032d-4c5d-8a73-855914d5d083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.270879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-be214e8f-032d-4c5d-8a73-855914d5d083 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:50.272125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-be214e8f-032d-4c5d-8a73-855914d5d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:50.310022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:50.311706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:50.341871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 822/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:50.344559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:50.372526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d320ac-d165-4cd0-99a4-c0cda95d860b service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:50.390629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:50.391324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.391604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.391881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.418648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:50.421161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:50.422657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:50.445078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 821/1643] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Jun 30 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:50.452299 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae625-b368-492b-b677-4636137c9442 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.453353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae625-b368-492b-b677-4636137c9442 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.453805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae625-b368-492b-b677-4636137c9442 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.460260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 822/1644] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jun 30 12:32:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:32:50.461931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.462390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.462819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.464057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 823/1645] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:50.490377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:50.492706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:50.519675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 823/1646] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:50.616106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:50.622101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:50.629164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466fc9-bac0-4831-8aa3-968ee5c9def0 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:50.637075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fb895-eebc-4ebc-91e4-0e55e27536d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:50.638469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fb895-eebc-4ebc-91e4-0e55e27536d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.638753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fb895-eebc-4ebc-91e4-0e55e27536d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.639025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fb895-eebc-4ebc-91e4-0e55e27536d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.664317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fb895-eebc-4ebc-91e4-0e55e27536d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:50.665850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fb895-eebc-4ebc-91e4-0e55e27536d9 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:50.694375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 824/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:50.847738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:50.862113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcdfb4-9d4a-4df5-90fa-ab1712f23380 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:50.876113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:50.879327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.879479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d47d49-5954-405a-afc9-05dc6a47f1f8 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.879758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d47d49-5954-405a-afc9-05dc6a47f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.880031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d47d49-5954-405a-afc9-05dc6a47f1f8 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.880345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.881055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.881474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 825/1648] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jun 30 12:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:32:50.896332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:50.896620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:50.896949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:50.969058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:50.986215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:51.025955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 826/1649] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:32:51.026816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:51.058740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:51.060221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:51.096045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 824/1650] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:51.106676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:51.123354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab849ca-b2b1-4bda-be0b-886f79923d96 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:51.130924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83b14e-797f-4b2d-b4f0-30362d56e250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:51.132102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83b14e-797f-4b2d-b4f0-30362d56e250 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:51.132253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83b14e-797f-4b2d-b4f0-30362d56e250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:51.132685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83b14e-797f-4b2d-b4f0-30362d56e250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:51.150896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83b14e-797f-4b2d-b4f0-30362d56e250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:51.154040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83b14e-797f-4b2d-b4f0-30362d56e250 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:51.171371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 825/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:51.239484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc60206-2401-4461-8480-67bb02bd6bfe None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:51.250547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:51.251623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:51.251767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:51.252005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] PATH_INFO: `/v3/users/da4fcf7be39e4b01a4c93e91b7cde4ba` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:51.253268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4fcf7be39e4b01a4c93e91b7cde4ba)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:51.257955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:51.370731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4fcf7be39e4b01a4c93e91b7cde4ba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.371011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4fcf7be39e4b01a4c93e91b7cde4ba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.396750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4fcf7be39e4b01a4c93e91b7cde4ba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.397538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4fcf7be39e4b01a4c93e91b7cde4ba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.419342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4fcf7be39e4b01a4c93e91b7cde4ba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.440325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 827/1652] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:51 2022] DELETE /identity/v3/users/da4fcf7be39e4b01a4c93e91b7cde4ba => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:51.444830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:51.445942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:51.446320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:51.497715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:51.503224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:51.536223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:51.537669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:51.703201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a4dd-e876-416d-b1b1-3377a720f8d8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:51.722422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:51.723342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:51.723695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:51.723902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] PATH_INFO: `/v3/users/04b8b5e97ba6419c999201b8699d0c47` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:51.734426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b8b5e97ba6419c999201b8699d0c47)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:51.736783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:51.756773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:51.762020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 826/1653] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:51.782571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:51.784509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:51.864970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48347bf2-8af9-4477-bef9-793692b5bf44 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:51.869499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897686-95b5-4a5c-8bb9-877d2e814c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:51.870397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897686-95b5-4a5c-8bb9-877d2e814c50 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:51.870832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897686-95b5-4a5c-8bb9-877d2e814c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:51.871218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897686-95b5-4a5c-8bb9-877d2e814c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:51.884429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03897686-95b5-4a5c-8bb9-877d2e814c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:51.886500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03897686-95b5-4a5c-8bb9-877d2e814c50 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:51.899727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 827/1654] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:51.910197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b8b5e97ba6419c999201b8699d0c47'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.910893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b8b5e97ba6419c999201b8699d0c47'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.953880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b8b5e97ba6419c999201b8699d0c47'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.954832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b8b5e97ba6419c999201b8699d0c47'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:51.973772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b8b5e97ba6419c999201b8699d0c47'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:52.011932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 828/1655] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:51 2022] DELETE /identity/v3/users/04b8b5e97ba6419c999201b8699d0c47 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:52.021963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:52.022416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:52.022917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:52.063581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:52.068532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:52.124171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:52.323518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 828/1656] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:52.398116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:52.399614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:52.559228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61694e06-1c04-4827-b7a1-efa65665e581 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:52.572790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4fb3b-bee1-4331-be87-c965d5b799db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:52.573455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4fb3b-bee1-4331-be87-c965d5b799db service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:52.573750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4fb3b-bee1-4331-be87-c965d5b799db service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:52.574224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4fb3b-bee1-4331-be87-c965d5b799db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:52.590034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4fb3b-bee1-4331-be87-c965d5b799db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:52.592054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4fb3b-bee1-4331-be87-c965d5b799db service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:52.607716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 829/1657] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:52.851525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:52.853203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:52.873731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5eb2a-ad4d-411f-ab1d-84c3e2a83086 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:52.901067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:52.908618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:52.909274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:52.909673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] PATH_INFO: `/v3/users/43d318f3788341c2b03adff2a77c0cd0` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:52.911428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d318f3788341c2b03adff2a77c0cd0)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:52.912910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: WARNING py.warnings [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:32:52.912910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: warnings.warn(msg) Jun 30 12:32:52.912910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: Jun 30 12:32:52.916071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:53.114573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d318f3788341c2b03adff2a77c0cd0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.116432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d318f3788341c2b03adff2a77c0cd0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.175644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d318f3788341c2b03adff2a77c0cd0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.176418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d318f3788341c2b03adff2a77c0cd0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.203586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d318f3788341c2b03adff2a77c0cd0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.224269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 829/1658] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:52 2022] DELETE /identity/v3/users/43d318f3788341c2b03adff2a77c0cd0 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:53.229048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:53.229463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:53.230533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:53.292997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:53.294847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:53.454846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 830/1659] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:53.557148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:53.558736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:53.682918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7f55f-0aa5-43c5-bda5-3fe0a82e09f9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:53.689736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:53.690643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:53.692191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:53.694094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:53.713417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:53.715874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:53.727079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:53.729108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:53.740523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 830/1660] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:53.745013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ed5af-cbe7-4a22-aa2d-6d5e8c7f7a2b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:53.769329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:53.772313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:53.772599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:53.775465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] PATH_INFO: `/v3/projects/f6f6e3f2d7c946a28515b911281a4d34` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:53.777359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6f6e3f2d7c946a28515b911281a4d34)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:53.779874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: WARNING py.warnings [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 12:32:53.779874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: warnings.warn(msg) Jun 30 12:32:53.779874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: Jun 30 12:32:53.782238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:53.852463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:53.905483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6f6e3f2d7c946a28515b911281a4d34', 'request_id': 'req-6613806e-0283-45c9-8b2b-6c9b684fc0f0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.948329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6f6e3f2d7c946a28515b911281a4d34', 'request_id': 'req-6613806e-0283-45c9-8b2b-6c9b684fc0f0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:53.975899 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6f6e3f2d7c946a28515b911281a4d34', 'request_id': 'req-6613806e-0283-45c9-8b2b-6c9b684fc0f0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.010404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] The token cache is being invalidate because project f6f6e3f2d7c946a28515b911281a4d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:32:54.011010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.012774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.015751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 831/1661] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:32:53 2022] DELETE /identity/v3/projects/f6f6e3f2d7c946a28515b911281a4d34 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:54.022740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:54.052422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8033e94-8e58-4def-a68c-cbcb9c2d88ee service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:54.137426 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:54.254884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:54.256051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:54.256472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:54.256903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] PATH_INFO: `/v3/users/f75572279a1f4560a63af18ad072b3c3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:54.260553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f75572279a1f4560a63af18ad072b3c3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:54.262789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:54.362687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f75572279a1f4560a63af18ad072b3c3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.363585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f75572279a1f4560a63af18ad072b3c3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.404440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f75572279a1f4560a63af18ad072b3c3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.407347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f75572279a1f4560a63af18ad072b3c3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.430046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-01931239-8c44-40ca-8109-9b0385503fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f75572279a1f4560a63af18ad072b3c3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.495673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 831/1662] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:54 2022] DELETE /identity/v3/users/f75572279a1f4560a63af18ad072b3c3 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:54.505657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:54.507006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:54.574368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613806e-0283-45c9-8b2b-6c9b684fc0f0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:54.745225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:54.748181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:54.748666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:54.749129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] PATH_INFO: `/v3/projects/e39c3bfdb43441b1a6241f5d926b62be` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:54.750669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39c3bfdb43441b1a6241f5d926b62be)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:54.753149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:54.893553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39c3bfdb43441b1a6241f5d926b62be', 'request_id': 'req-742f29a8-4ee2-48e3-932d-091e64c9d698'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.910130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39c3bfdb43441b1a6241f5d926b62be', 'request_id': 'req-742f29a8-4ee2-48e3-932d-091e64c9d698'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.920888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39c3bfdb43441b1a6241f5d926b62be', 'request_id': 'req-742f29a8-4ee2-48e3-932d-091e64c9d698'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.947173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] The token cache is being invalidate because project e39c3bfdb43441b1a6241f5d926b62be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:32:54.947854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.948804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:54.956728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 832/1663] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:32:54 2022] DELETE /identity/v3/projects/e39c3bfdb43441b1a6241f5d926b62be => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:54.961196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a93d3-ec44-4ab1-be72-727a6f204cc1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:54.962091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a93d3-ec44-4ab1-be72-727a6f204cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:54.962559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a93d3-ec44-4ab1-be72-727a6f204cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:54.964081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:54.970266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:55.008427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-971a93d3-ec44-4ab1-be72-727a6f204cc1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:55.026009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-971a93d3-ec44-4ab1-be72-727a6f204cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:55.046209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:55.093769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f29a8-4ee2-48e3-932d-091e64c9d698 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.100489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.101668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.102086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.102470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] PATH_INFO: `/v3/users/6dea5da5d96f435d8e8b0a572e3b2313` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.105590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dea5da5d96f435d8e8b0a572e3b2313)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.113623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.183478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 832/1664] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:55.191695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.192044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.192478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.229809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:55.231463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:55.235566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dea5da5d96f435d8e8b0a572e3b2313'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:55.238216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dea5da5d96f435d8e8b0a572e3b2313'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:55.256107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 833/1665] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:32:55.260745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:55.265946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dea5da5d96f435d8e8b0a572e3b2313'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:55.266980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dea5da5d96f435d8e8b0a572e3b2313'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:55.276680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bbfe0-f6b7-4213-831a-553a7b88688e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.288871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e959d4be-56dd-417b-ac58-69f8ff4b4dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dea5da5d96f435d8e8b0a572e3b2313'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:55.290541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.291335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.291690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.292617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.293631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:55.293844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.295635 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.324061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 833/1666] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:54 2022] DELETE /identity/v3/users/6dea5da5d96f435d8e8b0a572e3b2313 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:55.325759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.326067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.326345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.345708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 834/1667] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:32:55.356343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:55.369431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:55.370539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:55.382759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1bb36-0acd-4008-83b0-1d79815203bb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.461821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:55.523359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 834/1668] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:55.535104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.535822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.536106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.536381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.537181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.539319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.549797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:55.551934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:55.587341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 835/1669] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:55 2022] POST /identity/v3/projects => generated 357 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:55.600975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:55.641555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada9b6f-209d-438c-a190-954b84a09b30 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.663592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:55.730947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.733190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.733767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.734437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.774486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d953f5-a738-4ac2-aea9-4e453f1e1784 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.793464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.794188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.794471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.794739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.795837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.797749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.839235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.841819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.863156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 835/1670] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:55.866562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 836/1671] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:55 2022] POST /identity/v3/users => generated 279 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:32:55.871684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:55.886193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d35861-cfed-4e11-a64a-2f638f7c346a service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.897661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.897661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.897934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.898387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.899204 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.903438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.935195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 836/1672] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:55.936846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:55.949633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96ae25-296b-4bca-8b84-afe22cdc5c28 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:55.961431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:55.962205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:55.962989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:55.962989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:55.963424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:55.965236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:55.992547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 837/1673] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:55.999868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:56.003843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:56.007479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:56.013111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382558b-c207-448a-9e49-fc1a02f235ca admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:56.037430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b1f34-1e4e-4305-9dee-e7a9b5249adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:56.038363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1f34-1e4e-4305-9dee-e7a9b5249adb admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:56.038893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1f34-1e4e-4305-9dee-e7a9b5249adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:56.038893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1f34-1e4e-4305-9dee-e7a9b5249adb admin admin] PATH_INFO: `/v3/projects/9a9cceddd2ff4689bf553a6166a78b1a/users/5ee8513f64a644739709210f7803e031/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:56.042007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964b389-c435-46ed-9209-f16fddf3ed35 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:56.065048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:56.065958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:56.066221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:56.066490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] PATH_INFO: `/v3/projects/23c307d3f958455f81fd31898ab893ff` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:56.069119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c307d3f958455f81fd31898ab893ff)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:56.074099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:56.101690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b1f34-1e4e-4305-9dee-e7a9b5249adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9cceddd2ff4689bf553a6166a78b1a, user_id=5ee8513f64a644739709210f7803e031, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:56.104344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b1f34-1e4e-4305-9dee-e7a9b5249adb admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:56.129402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 837/1674] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:55 2022] PUT /identity/v3/projects/9a9cceddd2ff4689bf553a6166a78b1a/users/5ee8513f64a644739709210f7803e031/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:56.143042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0e1ad-2e51-42c6-8082-8a3144b06764 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:56.143413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0e1ad-2e51-42c6-8082-8a3144b06764 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:56.143669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0e1ad-2e51-42c6-8082-8a3144b06764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:56.202789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c307d3f958455f81fd31898ab893ff', 'request_id': 'req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.209214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6fa0e1ad-2e51-42c6-8082-8a3144b06764 None None] MFA Rules not processed for user `5ee8513f64a644739709210f7803e031`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:56.213367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6fa0e1ad-2e51-42c6-8082-8a3144b06764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:56.223113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c307d3f958455f81fd31898ab893ff', 'request_id': 'req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.230162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c307d3f958455f81fd31898ab893ff', 'request_id': 'req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.259265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] The token cache is being invalidate because project 23c307d3f958455f81fd31898ab893ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:32:56.259561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.260408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.265669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 838/1675] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:32:56 2022] DELETE /identity/v3/projects/23c307d3f958455f81fd31898ab893ff => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:56.271140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:56.273704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:56.345223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:56.415461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 838/1676] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:56.424911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:56.425288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:56.425528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:56.489751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca11a-fdca-45d7-a8be-cf0b789fa2da admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:56.520456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:56.521504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:56.523739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:56.524794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] PATH_INFO: `/v3/users/16a1b526126144ccae4669d3f4489ec4` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:56.527186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a1b526126144ccae4669d3f4489ec4)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:56.531517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] MFA Rules not processed for user `6ddb38b4d6864a6c96b84c9a4092eea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:56.532854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:56.533956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:56.572220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:56.674907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a1b526126144ccae4669d3f4489ec4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.676226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a1b526126144ccae4669d3f4489ec4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.693787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a1b526126144ccae4669d3f4489ec4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.694537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a1b526126144ccae4669d3f4489ec4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.711190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a1b526126144ccae4669d3f4489ec4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:56.743192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 839/1677] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:56 2022] DELETE /identity/v3/users/16a1b526126144ccae4669d3f4489ec4 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:56.744094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:56.761861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:56.798516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 839/1678] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:56.800765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:56.814980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:56.849289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d29aed-148d-49ea-9a8e-221e4049ac11 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:56.952469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81776839-3b86-42e7-a85f-94b74b3fd4f7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:56.968151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:56.968952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:56.969211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:56.969593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:56.970280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:56.975960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.008662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:57.011507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.012889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.013458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] PATH_INFO: `/v3/projects/1c29bb176b664eeca69c162be13cc1fc` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.017561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c29bb176b664eeca69c162be13cc1fc)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:57.024455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.048819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 840/1679] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:56 2022] POST /identity/v3/projects => generated 355 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:57.050057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:57.054005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.126881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:57.196914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47b85-eb26-4794-b079-a29d9a143da6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:57.209077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c29bb176b664eeca69c162be13cc1fc', 'request_id': 'req-553b37f0-d506-43b5-a0f7-80dd82ad49a0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.218006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c29bb176b664eeca69c162be13cc1fc', 'request_id': 'req-553b37f0-d506-43b5-a0f7-80dd82ad49a0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.225259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c29bb176b664eeca69c162be13cc1fc', 'request_id': 'req-553b37f0-d506-43b5-a0f7-80dd82ad49a0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.264772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] The token cache is being invalidate because project 1c29bb176b664eeca69c162be13cc1fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:32:57.265033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.267818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.270929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 840/1680] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:32:56 2022] DELETE /identity/v3/projects/1c29bb176b664eeca69c162be13cc1fc => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:57.271761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:57.278281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.306044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:57.308953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.309591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.310952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] PATH_INFO: `/v3/projects/162bbd0a5f74414e935a585d0ad5c09b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.314505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=162bbd0a5f74414e935a585d0ad5c09b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:57.319760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.398652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:57.449288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162bbd0a5f74414e935a585d0ad5c09b', 'request_id': 'req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.461383 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b37f0-d506-43b5-a0f7-80dd82ad49a0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:57.469840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162bbd0a5f74414e935a585d0ad5c09b', 'request_id': 'req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.485179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162bbd0a5f74414e935a585d0ad5c09b', 'request_id': 'req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.519485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] The token cache is being invalidate because project 162bbd0a5f74414e935a585d0ad5c09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:32:57.519878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.520788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.528107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 841/1681] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:32:57 2022] DELETE /identity/v3/projects/162bbd0a5f74414e935a585d0ad5c09b => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:57.529157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:57.530860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.620432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:57.621751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.621751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.621910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.627668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.647268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b45a5-bf12-47a5-aac7-fd7d34b009bb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:57.656615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ed321-9753-43d6-aa98-8b341806228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:57.657927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ed321-9753-43d6-aa98-8b341806228c admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.658631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ed321-9753-43d6-aa98-8b341806228c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.659192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ed321-9753-43d6-aa98-8b341806228c admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.660404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ed321-9753-43d6-aa98-8b341806228c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:57.663052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ed321-9753-43d6-aa98-8b341806228c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.676916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:57.729072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 842/1682] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:57 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:57.737040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.738650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.739418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.803101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:57.807973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.808784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:57.810984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.825565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 843/1683] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:57.827664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:57.829504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.852223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 841/1684] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:57.852325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a80a1b-7878-4f6d-8996-054449cdb00a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:57.852395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:57.852395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:57.860113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:57.863274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.866186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.866600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] PATH_INFO: `/v3/users/1428a95dde0347bb82d36b959da0aee9` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.867997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1428a95dde0347bb82d36b959da0aee9)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:57.870571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.881415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386043f5-3c34-4aa6-8722-0b147359af88 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:57.889919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:57.891099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:57.892488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:57.892882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] PATH_INFO: `/v3/projects/4d53b3e2a04f413982bdaae8a77fd35c` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:57.894302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d53b3e2a04f413982bdaae8a77fd35c)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:57.896324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:57.989107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1428a95dde0347bb82d36b959da0aee9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.990406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1428a95dde0347bb82d36b959da0aee9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:57.997662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d53b3e2a04f413982bdaae8a77fd35c', 'request_id': 'req-476ba316-aeb2-47d9-b90a-e354662ce556'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.013464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d53b3e2a04f413982bdaae8a77fd35c', 'request_id': 'req-476ba316-aeb2-47d9-b90a-e354662ce556'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.024275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1428a95dde0347bb82d36b959da0aee9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.029324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1428a95dde0347bb82d36b959da0aee9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.039386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d53b3e2a04f413982bdaae8a77fd35c', 'request_id': 'req-476ba316-aeb2-47d9-b90a-e354662ce556'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.053207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1428a95dde0347bb82d36b959da0aee9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.070972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:58.086570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 844/1685] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:32:57 2022] DELETE /identity/v3/users/1428a95dde0347bb82d36b959da0aee9 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:58.086726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] The token cache is being invalidate because project 4d53b3e2a04f413982bdaae8a77fd35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:32:58.086726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.086854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.087140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-476ba316-aeb2-47d9-b90a-e354662ce556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:32:58.089800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 842/1686] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:32:57 2022] DELETE /identity/v3/projects/4d53b3e2a04f413982bdaae8a77fd35c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:58.090563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:58.091683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390927-deee-47e3-ba96-30d21a221330 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.093360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390927-deee-47e3-ba96-30d21a221330 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.093954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c390927-deee-47e3-ba96-30d21a221330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.137136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5c390927-deee-47e3-ba96-30d21a221330 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:58.139879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5c390927-deee-47e3-ba96-30d21a221330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:58.257555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b06ca8-e20a-4355-8ab7-0f72d01d428d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.259363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 843/1687] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:58.265313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.265613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.265876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.275252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.276735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.277024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.277297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.278627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.284840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.315472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:58.319156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 845/1688] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:58.319156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.319411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:58.339399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e013cb1-2b9b-46bd-ba80-c0b3accd4758 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.342388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 844/1689] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:58.344332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.346230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.346954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:58.347243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.347642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.348036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.349197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:58.349646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.351428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5611-0587-4be3-90d1-54a6a58439ab admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.364529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 846/1690] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:58.366122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.366546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.366959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.386123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07254233-8722-42d6-b485-2b1ab6cc946d None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.387384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:58.388979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:58.396282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.397236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.397511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.397777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] PATH_INFO: `/v3/projects/90255a9629f54b43ab3f6850012ca071/users/95efa5fc30f7416cabf31a40c2a553e3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.400852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 847/1691] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:58.401858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.421492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bedc06-f8c3-48dc-a792-f50ec538cb24 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.426774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90255a9629f54b43ab3f6850012ca071, user_id=95efa5fc30f7416cabf31a40c2a553e3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.429130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.436380 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.437589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.438083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.438531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.439377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.443182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.454357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 845/1692] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:58 2022] PUT /identity/v3/projects/90255a9629f54b43ab3f6850012ca071/users/95efa5fc30f7416cabf31a40c2a553e3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:58.454543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.488453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 848/1693] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:58.489685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.519692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:58.610473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb966b39-8193-46c8-a0f5-a118ebe492f5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.626982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173faed7-9a6b-42ce-a7c6-5d478b948e88 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.632535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.633756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.633756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.634023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.634837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.640106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.666936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 849/1694] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:58.669928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.692064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faddd7-c050-4ae7-a335-887dbc6d6d19 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.707614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.709937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.710768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.712150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.714575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.720492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.755768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.759245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.759245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.759245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.759978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:32:58.760291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.761708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 850/1695] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:58.763735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.765666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.785796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335376-7940-4694-924e-09f43bc2b38e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.796489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 846/1696] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:32:58.801513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.805335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.806277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.806577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.806858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] PATH_INFO: `/v3/projects/90255a9629f54b43ab3f6850012ca071/users/95efa5fc30f7416cabf31a40c2a553e3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.812914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90255a9629f54b43ab3f6850012ca071, user_id=95efa5fc30f7416cabf31a40c2a553e3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.817103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.824916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f08b5d-1459-4f06-b50f-0a4b0e3ec67b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:58.838125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f42d2b-3d3b-4998-830e-74a5c6483767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:58.838622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f42d2b-3d3b-4998-830e-74a5c6483767 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.838906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f42d2b-3d3b-4998-830e-74a5c6483767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.839180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f42d2b-3d3b-4998-830e-74a5c6483767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:58.840001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f42d2b-3d3b-4998-830e-74a5c6483767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:58.865948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f42d2b-3d3b-4998-830e-74a5c6483767 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:58.874356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 851/1697] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:58 2022] PUT /identity/v3/projects/90255a9629f54b43ab3f6850012ca071/users/95efa5fc30f7416cabf31a40c2a553e3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:58.882903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:58.886930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:58.925600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 847/1698] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:58.928941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:58.929236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:58.929515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.014284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] MFA Rules not processed for user `95efa5fc30f7416cabf31a40c2a553e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:59.015885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:59.122132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd1cab-0580-455d-9af8-5e9e4c88e871 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.141655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.142757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.144997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.145444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.147114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.155686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.251102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 852/1699] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/projects => generated 387 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:59.252094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.285444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 848/1700] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:32:59.285444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.307856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954cb25-fd94-41d2-aca4-0f5ca977da70 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.398138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09790-65e9-41f9-8874-cba752071390 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.408556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.409369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.409685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.410829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.413012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.415852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.420219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:59.436263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.437109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.437455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.437794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.438760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.440803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.443033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 853/1701] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:59.446073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.449130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:59.493004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 849/1702] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:32:59 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:32:59.493004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.512920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a469f2-eb83-482c-9f81-bb1119c8d79e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.528059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.529115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.529667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.530074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] PATH_INFO: `/v3/projects/7ec36f714cb84a06ade75dcc926c47d1/users/d98f92cd6a01435fa604a5c60b6dafcb/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.556860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fd3ed-c010-44fa-8de3-2b9eda277fbc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.561381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec36f714cb84a06ade75dcc926c47d1, user_id=d98f92cd6a01435fa604a5c60b6dafcb, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.562539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758abd5-a4b9-45d9-b6f8-bd9d9602836a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.563269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758abd5-a4b9-45d9-b6f8-bd9d9602836a service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.563555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758abd5-a4b9-45d9-b6f8-bd9d9602836a service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.563829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758abd5-a4b9-45d9-b6f8-bd9d9602836a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.564402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.593777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 850/1703] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:59 2022] PUT /identity/v3/projects/7ec36f714cb84a06ade75dcc926c47d1/users/d98f92cd6a01435fa604a5c60b6dafcb/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:32:59.594758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.699192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:32:59.706297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758abd5-a4b9-45d9-b6f8-bd9d9602836a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.708659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758abd5-a4b9-45d9-b6f8-bd9d9602836a service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.724339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 854/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:32:59 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:32:59.726456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.726753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.727144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.731565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e467dc-030d-4c11-a1bd-4c53ae62b4a5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.743564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.744348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.744718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.745055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.747004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.748915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.765590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] MFA Rules not processed for user `d98f92cd6a01435fa604a5c60b6dafcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:32:59.768905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:32:59.777449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 851/1705] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:59.784029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.816839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0a606-3c2d-47c7-8c55-2fdc6562108a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.829935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.830779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.831121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.831539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.832432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.835266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:32:59.866104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 852/1706] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:32:59.866999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:32:59.917887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e62ec6-754c-4164-8f28-1725f82eb53c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:32:59.941370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:32:59.942294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:32:59.942571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:32:59.942839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] PATH_INFO: `/v3/projects/9635d9cc72f742b49d104efd71775778/users/d56538ea1461447dbe2134403ff3e84f/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:32:59.988396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9635d9cc72f742b49d104efd71775778, user_id=d56538ea1461447dbe2134403ff3e84f, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:32:59.991225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:00.012205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 855/1707] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:00.014917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:00.044829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 853/1708] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:32:59 2022] PUT /identity/v3/projects/9635d9cc72f742b49d104efd71775778/users/d56538ea1461447dbe2134403ff3e84f/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:00.046123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:00.189715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cd7c9-9009-44c7-8850-ae5c700acf07 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:00.207523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:00.208834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1f8f8-c7fb-42af-8117-235083c57dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:00.209707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1f8f8-c7fb-42af-8117-235083c57dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:00.210141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1f8f8-c7fb-42af-8117-235083c57dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:00.210552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1f8f8-c7fb-42af-8117-235083c57dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:00.211837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1f8f8-c7fb-42af-8117-235083c57dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:00.214005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1f8f8-c7fb-42af-8117-235083c57dcd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:00.271931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9675-c621-49b7-984b-c4816c4adf7c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:00.315374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 854/1709] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:00 2022] POST /identity/v3/projects => generated 381 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:00.316754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:00.317039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:00.317311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:00.403377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] MFA Rules not processed for user `d56538ea1461447dbe2134403ff3e84f`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:00.405211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:00.464941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:00.469126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:00.469950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:00.470770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:00.477981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:00.576508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:00.577839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:00.588370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 855/1710] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:00.589355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:00.590844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 856/1711] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:00 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:00.606165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:00.636796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123505b-9e31-4f3c-877c-88dae49900fe service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:00.647590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:00.736827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0199e-2b99-4eea-bfc6-341842ed4b78 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:00.753924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff99a6f-4dc0-4860-86e6-eaf5450a9634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:00.754209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99a6f-4dc0-4860-86e6-eaf5450a9634 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:00.754577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99a6f-4dc0-4860-86e6-eaf5450a9634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:00.754954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99a6f-4dc0-4860-86e6-eaf5450a9634 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:00.756025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff99a6f-4dc0-4860-86e6-eaf5450a9634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:00.757974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff99a6f-4dc0-4860-86e6-eaf5450a9634 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:00.813543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:00.815166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:00.815827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:00.816883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:00.819280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:00.827937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:00.852790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 856/1712] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:00 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:00.861852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:00.862149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:00.868942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:00.880409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 857/1713] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:00 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:00.882415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:00.886356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:00.926295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] MFA Rules not processed for user `3b688425466947198520ec7554bb49f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:00.928952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:00.973650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:01.058699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38b0e7-4650-441d-869c-5aa401321963 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.078560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.079506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.079747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.080020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] PATH_INFO: `/v3/projects/250c910483fe4ec89bedf5b52ae8f1d9` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.085588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250c910483fe4ec89bedf5b52ae8f1d9)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.087527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:01.090866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 857/1714] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:01.092922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.122408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82dcd-93f4-4adf-a3e6-465e25f97268 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.145843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.146543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.146850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.147253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.148387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.150639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:01.175969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 858/1715] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:01.182055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.195551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250c910483fe4ec89bedf5b52ae8f1d9', 'request_id': 'req-bff5fa85-263e-4fae-ba81-716abe836f55'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:01.200043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd3731-09b9-4f6d-9773-c51720956d4c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.219220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250c910483fe4ec89bedf5b52ae8f1d9', 'request_id': 'req-bff5fa85-263e-4fae-ba81-716abe836f55'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:01.239557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250c910483fe4ec89bedf5b52ae8f1d9', 'request_id': 'req-bff5fa85-263e-4fae-ba81-716abe836f55'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:01.274619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] The token cache is being invalidate because project 250c910483fe4ec89bedf5b52ae8f1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:01.274913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:01.276263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bff5fa85-263e-4fae-ba81-716abe836f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:01.285172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 858/1716] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:00 2022] DELETE /identity/v3/projects/250c910483fe4ec89bedf5b52ae8f1d9 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:01.285172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725721f2-790d-49bc-a0b0-74d4bdadf68f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.285172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725721f2-790d-49bc-a0b0-74d4bdadf68f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.285172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725721f2-790d-49bc-a0b0-74d4bdadf68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.320670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-725721f2-790d-49bc-a0b0-74d4bdadf68f None None] MFA Rules not processed for user `3b688425466947198520ec7554bb49f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:01.323495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-725721f2-790d-49bc-a0b0-74d4bdadf68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.356029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.356909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.357435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.357880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.358812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.361482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:01.424906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 859/1717] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:01 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:01.426511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.434422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.469238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b68222-5b94-4afc-8870-61e79c6878be admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.485737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.486828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.487169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.487535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] PATH_INFO: `/v3/projects/f44d5a4f0e8c476f8f1241abc2e64b59/users/c726e987e08b4e23bcdd03aeef7291bc/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.493970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 859/1718] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:01.495806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.495806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.496047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.530483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44d5a4f0e8c476f8f1241abc2e64b59, user_id=c726e987e08b4e23bcdd03aeef7291bc, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.533575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:01.541233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:01.542649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.567137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:01.574271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 860/1719] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:01 2022] PUT /identity/v3/projects/f44d5a4f0e8c476f8f1241abc2e64b59/users/c726e987e08b4e23bcdd03aeef7291bc/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:01.575262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.584564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 860/1720] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:01.585909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.586031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.586031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.736214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e14bff-ec05-4da8-928d-380874487931 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.751919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:01.753458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.754165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.754447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.754714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.755462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.757170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:01.786796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 861/1721] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:01.787776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.789251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020efcc-a956-4d05-926a-b5b7e403bdb6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.804895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08357c1d-db16-4b86-9f12-fe129d63b706 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.814203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6877db-6c50-43ad-99c4-cbe96e68ea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.815051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6877db-6c50-43ad-99c4-cbe96e68ea82 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.815431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6877db-6c50-43ad-99c4-cbe96e68ea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.815795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6877db-6c50-43ad-99c4-cbe96e68ea82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.816690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6877db-6c50-43ad-99c4-cbe96e68ea82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.820519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6877db-6c50-43ad-99c4-cbe96e68ea82 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:01.885180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 862/1722] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:01.888249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.888249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.888249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.924223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:01.925856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.932084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.932950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.933235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.934227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.936651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.937884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 863/1723] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:01.941101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:01.941688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:01.960548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e5edf-39f7-4edb-8e75-fbffc2cca426 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:01.971340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:01.973083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:01.974605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:01.974997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] PATH_INFO: `/v3/users/530fc1cea18b41818f95ef5462e9682f` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:01.977384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=530fc1cea18b41818f95ef5462e9682f)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:01.980240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.054171 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:02.057142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.072162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530fc1cea18b41818f95ef5462e9682f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:02.073248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530fc1cea18b41818f95ef5462e9682f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:02.083352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 861/1724] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:02.084418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:02.099375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530fc1cea18b41818f95ef5462e9682f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:02.102124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530fc1cea18b41818f95ef5462e9682f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:02.105148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd86f8c-8d0e-4ec2-a9e8-13d6752f22ea service nova] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:02.113809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:02.114574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.114975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.116121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.116346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:02.118205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.120538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530fc1cea18b41818f95ef5462e9682f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:02.145571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 862/1725] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:02.147734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:02.148813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 864/1726] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:01 2022] DELETE /identity/v3/users/530fc1cea18b41818f95ef5462e9682f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:02.151552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:02.154174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.155775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.183521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8f959-d487-433d-9b54-6c0fbaf40cb7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:02.299251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:02.335505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669c841-4070-4950-8a9c-8a8f60e9cc2f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:02.345319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:02.346077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.346359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.346634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.347383 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:02.350718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:02.353704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.386134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 865/1727] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:02.387124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:02.391378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.424276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7abb63-10e4-4a2d-bff0-147d00066957 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:02.427792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a33814-5304-46e7-a5d2-20b0a8f54f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:02.428991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a33814-5304-46e7-a5d2-20b0a8f54f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.429627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a33814-5304-46e7-a5d2-20b0a8f54f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.429756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a33814-5304-46e7-a5d2-20b0a8f54f3d admin admin] PATH_INFO: `/v3/projects/f44d5a4f0e8c476f8f1241abc2e64b59/users/c726e987e08b4e23bcdd03aeef7291bc/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.433917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a33814-5304-46e7-a5d2-20b0a8f54f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44d5a4f0e8c476f8f1241abc2e64b59, user_id=c726e987e08b4e23bcdd03aeef7291bc, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:02.436479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:02.437446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.438173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a33814-5304-46e7-a5d2-20b0a8f54f3d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.438357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.438680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] PATH_INFO: `/v3/projects/fa4024bf94444268a86966726891f09d/users/9bfb22266687427eaaed000d5f86aa57/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.462694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 863/1728] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:02 2022] PUT /identity/v3/projects/f44d5a4f0e8c476f8f1241abc2e64b59/users/c726e987e08b4e23bcdd03aeef7291bc/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:02.462694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee1414-bea0-4d18-a22b-1ef6b3a1c01e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.462694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee1414-bea0-4d18-a22b-1ef6b3a1c01e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.462694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee1414-bea0-4d18-a22b-1ef6b3a1c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.483507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4024bf94444268a86966726891f09d, user_id=9bfb22266687427eaaed000d5f86aa57, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:02.490282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.491825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-eeee1414-bea0-4d18-a22b-1ef6b3a1c01e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:02.493563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-eeee1414-bea0-4d18-a22b-1ef6b3a1c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.519685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:02.531877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 866/1729] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:02 2022] PUT /identity/v3/projects/fa4024bf94444268a86966726891f09d/users/9bfb22266687427eaaed000d5f86aa57/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:02.532826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:02.604701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 864/1730] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:02.614871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602a56c-9db0-46bb-a70c-0a70c8ee0bc6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.615366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602a56c-9db0-46bb-a70c-0a70c8ee0bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.615739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602a56c-9db0-46bb-a70c-0a70c8ee0bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.657726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7602a56c-9db0-46bb-a70c-0a70c8ee0bc6 None None] MFA Rules not processed for user `c726e987e08b4e23bcdd03aeef7291bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:02.659284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7602a56c-9db0-46bb-a70c-0a70c8ee0bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.737906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f04256-2491-406f-8586-676e05c46c7c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:02.756164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:02.757185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.757645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.758240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.759097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:02.761300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:02.791009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 865/1731] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:02.794303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.794726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.795167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:02.847128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] MFA Rules not processed for user `9bfb22266687427eaaed000d5f86aa57`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:02.853612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.858094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 867/1732] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:02 2022] POST /identity/v3/projects => generated 377 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:02.861449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:02.866501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:02.933462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f8f8-b608-4d94-b14f-f6fcf0ede1d1 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:02.942579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:02.943502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:02.943938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:02.944365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.024530 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:03.039495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.043313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.067766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 868/1733] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:03.068681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:03.070686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:03.076873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 866/1734] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:03.076873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:03.095974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ba5f9-678f-4550-8487-2189e0576932 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:03.110280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb6436-e53a-487b-95a7-be1db16e2a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:03.115266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb6436-e53a-487b-95a7-be1db16e2a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.115266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb6436-e53a-487b-95a7-be1db16e2a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.115266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb6436-e53a-487b-95a7-be1db16e2a45 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.115266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb6436-e53a-487b-95a7-be1db16e2a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.115266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb6436-e53a-487b-95a7-be1db16e2a45 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.156055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 867/1735] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:03 2022] POST /identity/v3/users => generated 289 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:03.158716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.161270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.161831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.215560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c423e9-83ea-4f83-a820-6a6c90d0e77d service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:03.225565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:03.226259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.226543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.226819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.238604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] MFA Rules not processed for user `26807e5e7aab425eb28e781289794265`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:03.243867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:03.258439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:03.356176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.372561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.407790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 869/1736] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:03.408743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:03.439055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 868/1737] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:03.448681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:03.451606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c87edb-cbbe-4b80-b8e2-cbbb19c72f0a service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:03.465139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f403e2-affc-45c7-9e45-b900819972de None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:03.473321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba6f91-0cc6-41f4-8428-507aecde59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:03.474040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba6f91-0cc6-41f4-8428-507aecde59ef admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.474372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba6f91-0cc6-41f4-8428-507aecde59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.474648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba6f91-0cc6-41f4-8428-507aecde59ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.475396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba6f91-0cc6-41f4-8428-507aecde59ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.477167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba6f91-0cc6-41f4-8428-507aecde59ef admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.484978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:03.485649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.485932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.487199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.487852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.491684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.500112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 869/1738] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:03.500112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd3d67-d2fd-4b61-b3ad-86be058bb057 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.500112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd3d67-d2fd-4b61-b3ad-86be058bb057 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.500112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd3d67-d2fd-4b61-b3ad-86be058bb057 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.504551 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 870/1739] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jun 30 12:33:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:33:03.513228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44785d-7d56-4720-bbd5-b15b78843e2d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.513923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44785d-7d56-4720-bbd5-b15b78843e2d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.514813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44785d-7d56-4720-bbd5-b15b78843e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.566953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 870/1740] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:03 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:03.567895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:03.573629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8b44785d-7d56-4720-bbd5-b15b78843e2d None None] MFA Rules not processed for user `26807e5e7aab425eb28e781289794265`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:03.577926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8b44785d-7d56-4720-bbd5-b15b78843e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:03.746115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 871/1741] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:03.750331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.751743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.752237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.802186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf00b-600c-4f09-a8bc-7b60e7df3ffd admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:03.817590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:03.818307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.818591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.818866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.819630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.826916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.858750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 871/1742] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:03.861606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:03.865359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] MFA Rules not processed for user `94b91e199622457787fa2a7e78d743c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:03.877661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:03.884925 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ddc04-aeeb-4b93-b989-af83be0102ae admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:03.895968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:03.896676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:03.896989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:03.897618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:03.900009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:03.913874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:03.979249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:03.991437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 872/1743] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:03 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:03.995825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:04.065545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 872/1744] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:04.066690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:04.068355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:04.170234 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:04.193250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819760e-10a2-4031-ab87-aeaf25096bcf admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:04.198743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:04.199431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.199752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.200034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.206147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:04.265386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c4d70-8428-46e7-b306-c39e3916b12a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:04.289122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:04.289818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.290099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.293707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.348177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:04.355404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:04.401792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 873/1745] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:04.404679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:04.417579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:04.423144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:04.448375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 873/1746] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:04.448573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb016e-2ba5-49d8-97c8-01280405189b service nova] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:04.464212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:04.468547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:04.473584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.473584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.473876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] PATH_INFO: `/v3/projects/555156e0df76492ea8eee7d8eca75a12/users/15513c3389ea4c2097c03e118b1d0d2b/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.500016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549e63b-4d35-4dc1-8479-883bff042140 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:04.522631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555156e0df76492ea8eee7d8eca75a12, user_id=15513c3389ea4c2097c03e118b1d0d2b, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:04.525225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:04.526071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:04.527400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.527894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.528360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.529741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:04.534278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:04.554577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 874/1747] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:04 2022] PUT /identity/v3/projects/555156e0df76492ea8eee7d8eca75a12/users/15513c3389ea4c2097c03e118b1d0d2b/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:04.555448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:04.565903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:04.573737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 874/1748] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:04.575222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:04.583395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:04.608991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d546186-904e-451f-b8c0-716edd1b93ac admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:04.789349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f126b0-5642-450e-9aef-23105142cfa1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:04.803153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:04.805640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.806007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.806993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] PATH_INFO: `/v3/projects/c7be5e06db6743c18915c8b326f275a9` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.808797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7be5e06db6743c18915c8b326f275a9)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:04.811920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:04.861315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742ea2c-1a6d-4bfb-a898-8a02a2b03bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:04.865974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ea2c-1a6d-4bfb-a898-8a02a2b03bdf service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.866268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ea2c-1a6d-4bfb-a898-8a02a2b03bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.866548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ea2c-1a6d-4bfb-a898-8a02a2b03bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.897370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ea2c-1a6d-4bfb-a898-8a02a2b03bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:04.899949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ea2c-1a6d-4bfb-a898-8a02a2b03bdf service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:04.923029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 875/1749] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:04.924835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:04.925124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:04.925415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:04.950701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7be5e06db6743c18915c8b326f275a9', 'request_id': 'req-d0584614-7552-4d07-b3c4-bea95d0447b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:04.963448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7be5e06db6743c18915c8b326f275a9', 'request_id': 'req-d0584614-7552-4d07-b3c4-bea95d0447b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:04.965291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] MFA Rules not processed for user `6ddb38b4d6864a6c96b84c9a4092eea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:04.966592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:04.983163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7be5e06db6743c18915c8b326f275a9', 'request_id': 'req-d0584614-7552-4d07-b3c4-bea95d0447b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:05.012173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] The token cache is being invalidate because project c7be5e06db6743c18915c8b326f275a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:05.012384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:05.014749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d0584614-7552-4d07-b3c4-bea95d0447b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:05.016435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:05.020698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 875/1750] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:04 2022] DELETE /identity/v3/projects/c7be5e06db6743c18915c8b326f275a9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:05.022044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac125224-6442-4f0c-849c-77e8d500caaa None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.024977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac125224-6442-4f0c-849c-77e8d500caaa None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.025356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac125224-6442-4f0c-849c-77e8d500caaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.038421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 876/1751] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:05.039859 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.041324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.070199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ac125224-6442-4f0c-849c-77e8d500caaa None None] MFA Rules not processed for user `15513c3389ea4c2097c03e118b1d0d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:05.071799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ac125224-6442-4f0c-849c-77e8d500caaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.092256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:05.124235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582536d-3421-40e2-921c-fa6a8f65a603 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:05.138725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf9611-c7f5-4a8b-9afe-f2ac1fda2d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:05.140118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf9611-c7f5-4a8b-9afe-f2ac1fda2d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.140487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf9611-c7f5-4a8b-9afe-f2ac1fda2d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.141173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf9611-c7f5-4a8b-9afe-f2ac1fda2d35 admin admin] PATH_INFO: `/v3/projects/e91d6cdcf166475089d246b736f2f852/users/d6688226568c45d09efebd2fb03e3a3f/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.169264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 876/1752] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:05.171083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e617c-d841-43aa-a767-46d6279e099f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.171576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e617c-d841-43aa-a767-46d6279e099f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.172059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e617c-d841-43aa-a767-46d6279e099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.186777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf9611-c7f5-4a8b-9afe-f2ac1fda2d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91d6cdcf166475089d246b736f2f852, user_id=d6688226568c45d09efebd2fb03e3a3f, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:05.191812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf9611-c7f5-4a8b-9afe-f2ac1fda2d35 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:05.203563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-043e617c-d841-43aa-a767-46d6279e099f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:05.205314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-043e617c-d841-43aa-a767-46d6279e099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.214763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 877/1753] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:05 2022] PUT /identity/v3/projects/e91d6cdcf166475089d246b736f2f852/users/d6688226568c45d09efebd2fb03e3a3f/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:05.219086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b2c65-d9d9-47a1-8b9f-6456600c86c2 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.219630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b2c65-d9d9-47a1-8b9f-6456600c86c2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.220112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b2c65-d9d9-47a1-8b9f-6456600c86c2 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.223120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 878/1754] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jun 30 12:33:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:33:05.225004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b2c65-d9d9-47a1-8b9f-6456600c86c2 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.229245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-266b2c65-d9d9-47a1-8b9f-6456600c86c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.347190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 877/1755] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:05.347190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e617c-d841-43aa-a767-46d6279e099f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.361887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e617c-d841-43aa-a767-46d6279e099f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:05.371423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d944a-850f-400e-8ca5-7172d8193ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:05.372227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d944a-850f-400e-8ca5-7172d8193ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.372498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d944a-850f-400e-8ca5-7172d8193ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.372807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d944a-850f-400e-8ca5-7172d8193ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.373919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d944a-850f-400e-8ca5-7172d8193ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:05.378518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d944a-850f-400e-8ca5-7172d8193ec8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:05.394983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b2c65-d9d9-47a1-8b9f-6456600c86c2 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:05.403616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 878/1756] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:05.405317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.405539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.405822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.410487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15d7c-1cd4-42c6-bb7e-4b7946e41f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:05.411193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15d7c-1cd4-42c6-bb7e-4b7946e41f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.411476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15d7c-1cd4-42c6-bb7e-4b7946e41f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.411750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15d7c-1cd4-42c6-bb7e-4b7946e41f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.412457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15d7c-1cd4-42c6-bb7e-4b7946e41f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:05.414433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15d7c-1cd4-42c6-bb7e-4b7946e41f47 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:05.444738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] MFA Rules not processed for user `94b91e199622457787fa2a7e78d743c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:05.445969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.461629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 879/1757] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:05 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:05.463254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.463546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.463867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.489887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:05.491194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.566525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 879/1758] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:05.569085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.596893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e41aa-4df5-45e9-94d9-02e2021451c1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:05.608088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:05.634520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 880/1759] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:05.635680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.652577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade877b-8577-4ba0-aa95-5d569a6a3cbb None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:05.684813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:05.686625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.686909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.687189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.687905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:05.695878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:05.740840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 881/1760] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:05 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:05.746751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.754495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:05.754811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:05.767820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:05.768974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:05.769302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:05.769636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] PATH_INFO: `/v3/projects/e91d6cdcf166475089d246b736f2f852/users/d6688226568c45d09efebd2fb03e3a3f/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:05.772598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91d6cdcf166475089d246b736f2f852, user_id=d6688226568c45d09efebd2fb03e3a3f, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:05.775329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:05.793531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 880/1761] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:05 2022] PUT /identity/v3/projects/e91d6cdcf166475089d246b736f2f852/users/d6688226568c45d09efebd2fb03e3a3f/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:05.795227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:05.871874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efa63d-126a-4967-b40e-05813abd53e7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.002686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.003917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.006174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.015544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e9c42-c0c8-4fc6-ac0d-bce683241358 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.021802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.026573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.031045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.031865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.032726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.033078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.036110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:06.036499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.038618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c0951-bf2d-4942-8d9e-bc77d994b161 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.053752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 882/1762] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:06.054884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:06.071689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 881/1763] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:06.073240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad6a37-dcd8-4f99-a492-54336cf61a19 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.073528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad6a37-dcd8-4f99-a492-54336cf61a19 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.073802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad6a37-dcd8-4f99-a492-54336cf61a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.078894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470dfda-1b87-4841-a949-d8e033a05cd9 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.090648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f76bd6-52c4-46c6-a1e8-b90ba1306674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.091345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f76bd6-52c4-46c6-a1e8-b90ba1306674 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.091628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f76bd6-52c4-46c6-a1e8-b90ba1306674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.092177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f76bd6-52c4-46c6-a1e8-b90ba1306674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.092765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f76bd6-52c4-46c6-a1e8-b90ba1306674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.094421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f76bd6-52c4-46c6-a1e8-b90ba1306674 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.129004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 883/1764] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:06.130218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcfc2-5dc4-4106-a8ff-8ab9732d9e85 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.130507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcfc2-5dc4-4106-a8ff-8ab9732d9e85 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.130783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcfc2-5dc4-4106-a8ff-8ab9732d9e85 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.149391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 884/1765] 10.222.0.57 () {54 vars in 960 bytes} [Thu Jun 30 12:33:06 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:33:06.149936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbcfc2-5dc4-4106-a8ff-8ab9732d9e85 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:06.153019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7bad6a37-dcd8-4f99-a492-54336cf61a19 None None] MFA Rules not processed for user `d6688226568c45d09efebd2fb03e3a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:06.154981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7bad6a37-dcd8-4f99-a492-54336cf61a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:06.172486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbcfc2-5dc4-4106-a8ff-8ab9732d9e85 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.182468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.183167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.187270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.187823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.188848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.191048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.269501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 885/1766] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:06 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:06.270646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:06.361308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 882/1767] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:06.362918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.363308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.363869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.446084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:06.456603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:06.475282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59fba1-a5d1-44a8-81aa-502ab2e34ca3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.490887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.491894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.492567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.492901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] PATH_INFO: `/v3/projects/243a2af6952540ac8cc6c31bf7cdfb8e/users/0a1e1d2ddd8e4463b4254e7c8bb96950/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.522851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:06.534099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243a2af6952540ac8cc6c31bf7cdfb8e, user_id=0a1e1d2ddd8e4463b4254e7c8bb96950, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.537109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.581454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 886/1768] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:06 2022] PUT /identity/v3/projects/243a2af6952540ac8cc6c31bf7cdfb8e/users/0a1e1d2ddd8e4463b4254e7c8bb96950/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:06.581454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:06.607002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 883/1769] 10.222.0.57 () {58 vars in 1057 bytes} [Thu Jun 30 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:06.609662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:06.625124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196941f-a757-4f4f-a43f-d26cc2591230 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.632249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:06.778334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.779238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.779678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.779958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.782230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5c8e-fb04-4357-aa0f-b86ca29ab008 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:06.782833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.789119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.814333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:06.815027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:06.815302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:06.815576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:06.816454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:06.818444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:06.839180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 884/1770] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:06.839717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:06.847707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:06.896089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 887/1771] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:06 2022] POST /identity/v3/users => generated 303 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:06.902131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:07.023419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cca1f-8ed7-4164-97df-aa70d661237c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:07.035282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:07.035990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.036267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.036550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.115249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303bcf7-33a9-4431-b14f-c328c518a4c6 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:07.144284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:07.150109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.150498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.152267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.161149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:07.205541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:07.207361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:07.253836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 888/1772] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:07.260348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:07.279939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:07.281447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:07.288055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500ad19-e134-4ff7-87ca-e83d16e8e6eb service placement] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:07.311505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f411-6dba-4bbd-a723-d65b99773215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:07.312515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f411-6dba-4bbd-a723-d65b99773215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.312745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f411-6dba-4bbd-a723-d65b99773215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.313023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f411-6dba-4bbd-a723-d65b99773215 admin admin] PATH_INFO: `/v3/projects/243a2af6952540ac8cc6c31bf7cdfb8e/users/0a1e1d2ddd8e4463b4254e7c8bb96950/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.320580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 885/1773] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:07.321478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:07.323006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9f411-6dba-4bbd-a723-d65b99773215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243a2af6952540ac8cc6c31bf7cdfb8e, user_id=0a1e1d2ddd8e4463b4254e7c8bb96950, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:07.333112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9f411-6dba-4bbd-a723-d65b99773215 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:07.338327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95e746-7d6d-4388-8dd7-d684fb122be4 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:07.352298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:07.352870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.353163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.353443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.354205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:07.355818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:07.377568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 889/1774] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:07 2022] PUT /identity/v3/projects/243a2af6952540ac8cc6c31bf7cdfb8e/users/0a1e1d2ddd8e4463b4254e7c8bb96950/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:07.379421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.379759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.379980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.386555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 886/1775] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:07.389005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:07.423907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4ce0-1b09-415d-a47b-4507837a4a12 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:07.449978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] MFA Rules not processed for user `0a1e1d2ddd8e4463b4254e7c8bb96950`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:07.451387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:07.567010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:07.610579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:07.630710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:07.631512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.631780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.632468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.632851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:07.636474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:07.675450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 887/1776] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:07.693693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:07.719351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 890/1777] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:07.741405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913c90a-308e-4f66-8e4c-f9c302ae8c68 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:07.760524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abce33b-cfb6-4a2f-8c55-e5e9980735e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:07.761719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce33b-cfb6-4a2f-8c55-e5e9980735e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.762006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce33b-cfb6-4a2f-8c55-e5e9980735e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.762310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce33b-cfb6-4a2f-8c55-e5e9980735e1 admin admin] PATH_INFO: `/v3/projects/2d02840806084fb3961aafcc20083542/users/4f8a9b89fb7a4a0ea4f76cf4506a80df/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.778862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:07.834756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abce33b-cfb6-4a2f-8c55-e5e9980735e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d02840806084fb3961aafcc20083542, user_id=4f8a9b89fb7a4a0ea4f76cf4506a80df, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:07.838959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abce33b-cfb6-4a2f-8c55-e5e9980735e1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:07.879900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 888/1778] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:07 2022] PUT /identity/v3/projects/2d02840806084fb3961aafcc20083542/users/4f8a9b89fb7a4a0ea4f76cf4506a80df/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:07.881559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:07.882227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:07.883935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:07.934560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:07.946620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:07.956288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef64d66-631c-4870-ae0d-bd527224c22b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:08.075581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 889/1779] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:08.085053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:08.090061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:08.105117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042aebc1-13e7-4f1c-97ad-fc045b9ee98c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:08.105870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042aebc1-13e7-4f1c-97ad-fc045b9ee98c service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:08.106150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042aebc1-13e7-4f1c-97ad-fc045b9ee98c service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:08.106429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042aebc1-13e7-4f1c-97ad-fc045b9ee98c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:08.386986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c34d34-8f23-4610-afd0-d991be25b091 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:08.405607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:08.408608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:08.412786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:08.413197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:08.413583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042aebc1-13e7-4f1c-97ad-fc045b9ee98c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:08.416539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042aebc1-13e7-4f1c-97ad-fc045b9ee98c service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:08.429492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 891/1780] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:08.431157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00864f-ff5c-4d92-977a-547b63b3da2f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:08.431585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00864f-ff5c-4d92-977a-547b63b3da2f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:08.432110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00864f-ff5c-4d92-977a-547b63b3da2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:08.447442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:08.450139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:08.455441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ca00864f-ff5c-4d92-977a-547b63b3da2f None None] MFA Rules not processed for user `26807e5e7aab425eb28e781289794265`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:08.459833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ca00864f-ff5c-4d92-977a-547b63b3da2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:08.523241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 890/1781] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:08.667618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:08.703889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 892/1782] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:09.685755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:09.687863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:09.712447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2406f8-2553-485b-bef4-f602472655b9 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:09.722095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:09.724117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:09.725796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:09.726155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:09.730765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:09.877259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:09.897278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:09.901172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:09.923223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 891/1783] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:10.267185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.267713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.267788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.306992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] MFA Rules not processed for user `4f8a9b89fb7a4a0ea4f76cf4506a80df`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:10.311318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:10.422257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 893/1784] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:10.427007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:10.447444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300bfb8-91b2-40a0-8abf-20366b9e2663 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:10.453958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:10.454653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.454937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.455213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.455942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:10.457952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:10.489840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 892/1785] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:10 2022] POST /identity/v3/projects => generated 403 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:10.491526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:10.503840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5270946-4687-4aad-b8b2-cd9ef1ad56d2 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:10.570131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:10.572439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.572756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.573033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.573769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:10.575804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:10.611680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 894/1786] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:10 2022] POST /identity/v3/users => generated 308 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:10.613454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:10.630858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abfae7-8add-4b13-8fdb-d757b5d19cd3 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:10.635895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:10.636664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.636980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.637262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.638036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:10.639709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:10.651384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 893/1787] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:10.656042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:10.666199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64332362-3e3a-4146-87d8-8f04bd617bae admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:10.670879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:10.671785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.672276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.672553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] PATH_INFO: `/v3/projects/9eb7d522c41b4b4a96b3fcfa0e0618e6/users/fa209f0f24e54231ad40ab168bf41687/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.690672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb7d522c41b4b4a96b3fcfa0e0618e6, user_id=fa209f0f24e54231ad40ab168bf41687, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:10.693444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:10.707643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 895/1788] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:10 2022] PUT /identity/v3/projects/9eb7d522c41b4b4a96b3fcfa0e0618e6/users/fa209f0f24e54231ad40ab168bf41687/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:10.712200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:10.724168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1fcf9-efd3-4c16-837f-97ae812442c9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:10.781791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370f4d-d4c3-4e20-95e0-6736c7c9830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:10.782468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370f4d-d4c3-4e20-95e0-6736c7c9830d admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.782750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370f4d-d4c3-4e20-95e0-6736c7c9830d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.783028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370f4d-d4c3-4e20-95e0-6736c7c9830d admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.783754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60370f4d-d4c3-4e20-95e0-6736c7c9830d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:10.785409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60370f4d-d4c3-4e20-95e0-6736c7c9830d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:10.795208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 894/1789] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:10.802477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:10.819985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ad009-02d1-46f9-ace0-7c21003b9582 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:10.827961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac8d10-6f0f-49bd-83e0-b89e9e9ea484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:10.828908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac8d10-6f0f-49bd-83e0-b89e9e9ea484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:10.829220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac8d10-6f0f-49bd-83e0-b89e9e9ea484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:10.829604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac8d10-6f0f-49bd-83e0-b89e9e9ea484 admin admin] PATH_INFO: `/v3/projects/9eb7d522c41b4b4a96b3fcfa0e0618e6/users/fa209f0f24e54231ad40ab168bf41687/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:10.831739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac8d10-6f0f-49bd-83e0-b89e9e9ea484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb7d522c41b4b4a96b3fcfa0e0618e6, user_id=fa209f0f24e54231ad40ab168bf41687, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:10.834442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac8d10-6f0f-49bd-83e0-b89e9e9ea484 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:10.845940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 896/1790] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:10 2022] PUT /identity/v3/projects/9eb7d522c41b4b4a96b3fcfa0e0618e6/users/fa209f0f24e54231ad40ab168bf41687/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:11.519108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4191-84fe-4d63-86f0-c31b710dcf1c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:11.519378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4191-84fe-4d63-86f0-c31b710dcf1c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:11.519657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4191-84fe-4d63-86f0-c31b710dcf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:11.546402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-444d4191-84fe-4d63-86f0-c31b710dcf1c None None] MFA Rules not processed for user `fa209f0f24e54231ad40ab168bf41687`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:11.547742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-444d4191-84fe-4d63-86f0-c31b710dcf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:11.624529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 895/1791] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:12.348112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb79f-6ddf-4465-b9e6-437acfccbf78 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:12.348647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb79f-6ddf-4465-b9e6-437acfccbf78 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:12.348856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb79f-6ddf-4465-b9e6-437acfccbf78 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:12.350635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 897/1792] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:33:12.366836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:12.367411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:12.367411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:12.405578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:12.409695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:12.481840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:12.533566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 896/1793] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:12.553622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeb79f-6ddf-4465-b9e6-437acfccbf78 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:12.572609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeb79f-6ddf-4465-b9e6-437acfccbf78 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:12.580366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:12.581667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:12.582125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:12.582602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:12.595053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:12.597639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:12.621952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 898/1794] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:13.401059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:13.544379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa828ae-7aff-4aae-a55c-2cf115a79850 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:13.553484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:13.556425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:13.556982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:13.557483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] PATH_INFO: `/v3/users/4f8a9b89fb7a4a0ea4f76cf4506a80df` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:13.559141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8a9b89fb7a4a0ea4f76cf4506a80df)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:13.563549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:13.653831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8a9b89fb7a4a0ea4f76cf4506a80df'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:13.655256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8a9b89fb7a4a0ea4f76cf4506a80df'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:13.670214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8a9b89fb7a4a0ea4f76cf4506a80df'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:13.671310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8a9b89fb7a4a0ea4f76cf4506a80df'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:13.711860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c34546c1-5145-4805-969b-ecff457d7dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8a9b89fb7a4a0ea4f76cf4506a80df'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:13.742262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 897/1795] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:13 2022] DELETE /identity/v3/users/4f8a9b89fb7a4a0ea4f76cf4506a80df => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:14.418193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:14.435553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563d8ea-2a05-4187-955b-49c0252532d7 service placement] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:14.550051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:14.681907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:14.682397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:14.682579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:14.683140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] PATH_INFO: `/v3/projects/2d02840806084fb3961aafcc20083542` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:14.684503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d02840806084fb3961aafcc20083542)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:14.686760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:14.691559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:14.692099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:14.692307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:14.746148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:14.752423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:14.853306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d02840806084fb3961aafcc20083542', 'request_id': 'req-d59949f2-a700-464f-b54b-753d2a1ddeef'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:14.868703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d02840806084fb3961aafcc20083542', 'request_id': 'req-d59949f2-a700-464f-b54b-753d2a1ddeef'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:14.899195 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d02840806084fb3961aafcc20083542', 'request_id': 'req-d59949f2-a700-464f-b54b-753d2a1ddeef'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:14.935605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] The token cache is being invalidate because project 2d02840806084fb3961aafcc20083542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:14.935944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:14.937890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:14.950326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 899/1796] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:14 2022] DELETE /identity/v3/projects/2d02840806084fb3961aafcc20083542 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:14.956289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:14.960438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:15.051133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 898/1797] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:15.291674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59949f2-a700-464f-b54b-753d2a1ddeef admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:15.307454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:15.309630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:15.311881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:15.315289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] PATH_INFO: `/v3/users/fa209f0f24e54231ad40ab168bf41687` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:15.319293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa209f0f24e54231ad40ab168bf41687)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:15.329698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:15.444002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:15.445876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:15.456032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:15.467262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa209f0f24e54231ad40ab168bf41687'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:15.471314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa209f0f24e54231ad40ab168bf41687'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:15.489585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:15.507853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa209f0f24e54231ad40ab168bf41687'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:15.508876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa209f0f24e54231ad40ab168bf41687'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:15.537807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa209f0f24e54231ad40ab168bf41687'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:15.570675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 900/1798] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:14 2022] DELETE /identity/v3/users/fa209f0f24e54231ad40ab168bf41687 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:15.661599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad0516-bcd0-4801-aefd-8179c1666814 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:15.758721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:15.760403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:15.760879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:15.761488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:15.763940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:15.856656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:15.858549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b22da-8df4-43b2-b880-a633e53c6b06 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:15.870756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 899/1799] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:33:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 12:33:16.172656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:16.174576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:16.361758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf35e-0232-4d1a-9864-4faedb94d231 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:16.383250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:16.383250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.383250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.383250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] PATH_INFO: `/v3/projects/9eb7d522c41b4b4a96b3fcfa0e0618e6` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.384425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb7d522c41b4b4a96b3fcfa0e0618e6)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:16.386527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:16.463066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb7d522c41b4b4a96b3fcfa0e0618e6', 'request_id': 'req-04480975-814f-4bcf-987a-fde8459400c0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:16.470101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb7d522c41b4b4a96b3fcfa0e0618e6', 'request_id': 'req-04480975-814f-4bcf-987a-fde8459400c0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:16.476281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb7d522c41b4b4a96b3fcfa0e0618e6', 'request_id': 'req-04480975-814f-4bcf-987a-fde8459400c0'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:16.492414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] The token cache is being invalidate because project 9eb7d522c41b4b4a96b3fcfa0e0618e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:16.492735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:16.493727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-04480975-814f-4bcf-987a-fde8459400c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:16.495526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 901/1800] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:16 2022] DELETE /identity/v3/projects/9eb7d522c41b4b4a96b3fcfa0e0618e6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:16.500878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.501246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.501608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.520871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:16.522127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:16.580014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:16.612683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 900/1801] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:16.616337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.617492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.617660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.701670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:16.701670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:16.721847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 902/1802] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:16.723294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:16.735205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283775-817d-45bb-849a-01c8f0f51ea9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:16.741524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:16.742216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.742500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.742772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.743519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:16.743827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:16.745474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:16.792432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 901/1803] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:16.794089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:16.807188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debcb18-1fb4-4acf-8ed6-0ed734cab79d None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:16.813127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:16.813813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.814093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.814361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.815111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:16.817255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:16.850171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 903/1804] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:16 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:16.854586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:16.866451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710117-e73f-4368-966b-33bc7dbae952 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:16.928233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:16.928983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.929298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.929572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.930268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:16.932230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:16.954607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 902/1805] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:16 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:16.958850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:16.971868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6c94e-7d63-438a-85df-2df9c4614004 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:16.976846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:16.977752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:16.978036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:16.978315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:16.979377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:16.980989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:16.994594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 904/1806] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:16.999379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.009789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847743f-795d-4fba-ad5a-fdba6e66ed0e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.015382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aeebb-076b-4b53-a0e0-a1784d7f7482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.016178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aeebb-076b-4b53-a0e0-a1784d7f7482 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.016585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aeebb-076b-4b53-a0e0-a1784d7f7482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.016944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aeebb-076b-4b53-a0e0-a1784d7f7482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.017770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55aeebb-076b-4b53-a0e0-a1784d7f7482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.019455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55aeebb-076b-4b53-a0e0-a1784d7f7482 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.036977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 903/1807] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:17.040158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.051981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111a13-13c3-4d2a-908e-d3469ce2a8db admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.057000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.057907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.058189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.058462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] PATH_INFO: `/v3/projects/163e2ec49f5d45d8a66c5c85d261a8e1/users/8edd30cd3c384935b6f8c64734fc9ba3/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.073739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163e2ec49f5d45d8a66c5c85d261a8e1, user_id=8edd30cd3c384935b6f8c64734fc9ba3, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.076552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.091287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 905/1808] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:17 2022] PUT /identity/v3/projects/163e2ec49f5d45d8a66c5c85d261a8e1/users/8edd30cd3c384935b6f8c64734fc9ba3/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:17.102996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.103391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.103779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.136975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] MFA Rules not processed for user `8edd30cd3c384935b6f8c64734fc9ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:17.136975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:17.199550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 904/1809] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:17.204183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.208883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:17.271539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe93a9-1c8d-4004-9435-06d33f1124f4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.276508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.277214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.277498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.277773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.278471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.280593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.312515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 906/1810] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:17 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:17.317482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.331208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab552c34-b27d-4412-8ca7-28a0cd1acc24 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.374517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:17.399592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.401621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.401621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.401621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.403134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.405346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.441843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 905/1811] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:17 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:17.443391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.457893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65ea24-b2e7-4127-b4d1-45697ce83cc9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.466960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.468043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.468456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.468915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.469801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.471725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.490973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 907/1812] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:17.492039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.504645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c9345-1336-471d-b6f5-234657a7bad0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.512489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.513993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.514494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.514915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] PATH_INFO: `/v3/projects/928e8f9d41444147a1325fffc94858db/users/347de8d8136a49a3bfae62fb5daa7809/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.534593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=928e8f9d41444147a1325fffc94858db, user_id=347de8d8136a49a3bfae62fb5daa7809, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.537680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.569159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 906/1813] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:17 2022] PUT /identity/v3/projects/928e8f9d41444147a1325fffc94858db/users/347de8d8136a49a3bfae62fb5daa7809/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:17.575573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.588839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47020bbc-ed16-4e92-aeb2-84907f510b70 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.664822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe731327-c68d-48b0-b9f3-d518a906a47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.665510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731327-c68d-48b0-b9f3-d518a906a47d admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.665786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731327-c68d-48b0-b9f3-d518a906a47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.666058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731327-c68d-48b0-b9f3-d518a906a47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.666926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe731327-c68d-48b0-b9f3-d518a906a47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.671090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe731327-c68d-48b0-b9f3-d518a906a47d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.683592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 908/1814] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:17.689036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.701912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3657c-2524-42e3-9968-9514bdc194cc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.707189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.708308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.708590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.708890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] PATH_INFO: `/v3/projects/928e8f9d41444147a1325fffc94858db/users/347de8d8136a49a3bfae62fb5daa7809/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.710948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=928e8f9d41444147a1325fffc94858db, user_id=347de8d8136a49a3bfae62fb5daa7809, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:17.714016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:17.744308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 907/1815] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:17 2022] PUT /identity/v3/projects/928e8f9d41444147a1325fffc94858db/users/347de8d8136a49a3bfae62fb5daa7809/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:17.755016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.755313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.755593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.793770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] MFA Rules not processed for user `347de8d8136a49a3bfae62fb5daa7809`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:17.798067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:17.835744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:17.848507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:17.960417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60bb3e-d0b7-4d51-8726-6f7abfe8eccd admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:17.970160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:17.970855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:17.971547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:17.971884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:17.974263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:17.994895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:18.012657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 909/1816] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:18.027144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:18.031893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:18.065445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ccbe1-e847-417a-b616-a1e26b98fb22 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:18.077673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:18.078352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:18.078571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:18.078979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:18.097287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:18.098901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:18.099108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:18.100866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:18.110948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 910/1817] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:18.126612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 908/1818] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:18.239095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:18.252676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041fe04-2197-4583-b840-6aedbed7614d service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:18.262065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:18.266734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:18.266734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:18.266734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:18.409570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:18.411269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:18.439901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 911/1819] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:19.549143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:19.566675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ed7da-fe63-491d-a638-2b642174c51b service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:19.572270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:19.573024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:19.573480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:19.573906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:19.577546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:19.803305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:19.826545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:19.828032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516c072-8402-4af8-ac3b-b6ad2629f070 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:19.838782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 909/1820] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:19.876590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:19.878480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:20.020453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8b02-1e62-4bec-9dae-a7fb78970eb7 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:20.032096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:20.033156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:20.033441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:20.033715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] PATH_INFO: `/v3/users/d98f92cd6a01435fa604a5c60b6dafcb` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:20.034968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98f92cd6a01435fa604a5c60b6dafcb)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:20.037257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:20.147483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98f92cd6a01435fa604a5c60b6dafcb'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:20.148429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98f92cd6a01435fa604a5c60b6dafcb'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:20.196066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98f92cd6a01435fa604a5c60b6dafcb'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:20.197349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98f92cd6a01435fa604a5c60b6dafcb'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:20.226905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98f92cd6a01435fa604a5c60b6dafcb'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:20.260396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 912/1821] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:19 2022] DELETE /identity/v3/users/d98f92cd6a01435fa604a5c60b6dafcb => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:20.266859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:20.268839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:20.268839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:20.297896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:20.299772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:20.453718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 910/1822] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:20.472440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:20.474678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:20.490657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:20.638562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca08b01-9226-4da2-92e0-d1b278f2b6c8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:20.644604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:20.645428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:20.645807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:20.646149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:20.676185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:20.677884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:20.694445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 913/1823] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:21.078259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:21.080399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:21.096794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea66bf-98e7-41de-9c32-abc0ffb9948e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:21.101729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:21.102613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:21.102884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:21.103156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] PATH_INFO: `/v3/projects/7ec36f714cb84a06ade75dcc926c47d1` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:21.104948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec36f714cb84a06ade75dcc926c47d1)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:21.106784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:21.158993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec36f714cb84a06ade75dcc926c47d1', 'request_id': 'req-ea0eb14f-c278-40a2-8fb1-9c5567de9627'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.174478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec36f714cb84a06ade75dcc926c47d1', 'request_id': 'req-ea0eb14f-c278-40a2-8fb1-9c5567de9627'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.181060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec36f714cb84a06ade75dcc926c47d1', 'request_id': 'req-ea0eb14f-c278-40a2-8fb1-9c5567de9627'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.193079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] The token cache is being invalidate because project 7ec36f714cb84a06ade75dcc926c47d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:21.193545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.194458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.199151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 911/1824] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:21 2022] DELETE /identity/v3/projects/7ec36f714cb84a06ade75dcc926c47d1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:21.200514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:21.212734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607f744-5622-4527-bd1c-2c5136373f48 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:21.300375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:21.304170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:21.304741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:21.305177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] PATH_INFO: `/v3/users/c726e987e08b4e23bcdd03aeef7291bc` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:21.309660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c726e987e08b4e23bcdd03aeef7291bc)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:21.312454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:21.370159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c726e987e08b4e23bcdd03aeef7291bc'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.370911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c726e987e08b4e23bcdd03aeef7291bc'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.385050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c726e987e08b4e23bcdd03aeef7291bc'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.385812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c726e987e08b4e23bcdd03aeef7291bc'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.394809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-7dcc20cd-a55f-4165-8256-b22d69535af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c726e987e08b4e23bcdd03aeef7291bc'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.408363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 914/1825] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:21 2022] DELETE /identity/v3/users/c726e987e08b4e23bcdd03aeef7291bc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:21.735812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:21.737619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:21.751368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0eb14f-c278-40a2-8fb1-9c5567de9627 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:21.861703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:21.863084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:21.863516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:21.863873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] PATH_INFO: `/v3/projects/f44d5a4f0e8c476f8f1241abc2e64b59` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:21.865550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44d5a4f0e8c476f8f1241abc2e64b59)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:21.868153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:21.884951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:21.956600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44d5a4f0e8c476f8f1241abc2e64b59', 'request_id': 'req-5fa012ab-588e-4a88-a476-b08fd85d01b8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.970151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44d5a4f0e8c476f8f1241abc2e64b59', 'request_id': 'req-5fa012ab-588e-4a88-a476-b08fd85d01b8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.976517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44d5a4f0e8c476f8f1241abc2e64b59', 'request_id': 'req-5fa012ab-588e-4a88-a476-b08fd85d01b8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.991258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] The token cache is being invalidate because project f44d5a4f0e8c476f8f1241abc2e64b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:21.991593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.992265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5fa012ab-588e-4a88-a476-b08fd85d01b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:21.993992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 912/1826] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:21 2022] DELETE /identity/v3/projects/f44d5a4f0e8c476f8f1241abc2e64b59 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:22.000068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.000433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.000821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.038044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:22.042236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:22.078899 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:22.180570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 915/1827] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:22.189772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.190363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.190513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.214650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:22.215874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:22.243468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 913/1828] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:22.254732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.264082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b745c-141a-49d7-9c9d-1e10dec4bf57 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.271552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.273152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.273517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.275466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.276392 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:22.276822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.279082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.292993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 916/1829] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:22.297576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.307450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc473b-1d70-424f-9277-492c64900d90 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.315906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.316671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.316958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.317232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.317941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.320950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.358946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 914/1830] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:22 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:22.361884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.373769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e44b8-e219-4815-84df-5ba3f30cc1e8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.445718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.446643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.447069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.447487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.448362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.451179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.483929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 917/1831] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:22 2022] POST /identity/v3/users => generated 305 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:22.488974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.516861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e62d6-99af-4074-a4d3-b7a2a719dadc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.523873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.524584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.524961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.525238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.526100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.527796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.540669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 915/1832] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:22.544714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.563440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708e2b2-4a7e-47bc-b59b-bb160ead7f85 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.575506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc69607-4ee5-423d-909e-a4502c0f023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.578021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc69607-4ee5-423d-909e-a4502c0f023f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.578568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc69607-4ee5-423d-909e-a4502c0f023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.579381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc69607-4ee5-423d-909e-a4502c0f023f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.580414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc69607-4ee5-423d-909e-a4502c0f023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.582409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc69607-4ee5-423d-909e-a4502c0f023f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.600070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 918/1833] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:22.601625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.615185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342528-a16f-47d7-a5aa-5dc320ecaef9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.632057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.633276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.633784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.634155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] PATH_INFO: `/v3/projects/790a398668664cdb83089761d62a0cc6/users/8cfd60845f984b0591ac03c53003a06b/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.654099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=790a398668664cdb83089761d62a0cc6, user_id=8cfd60845f984b0591ac03c53003a06b, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.656847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.672670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 916/1834] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:22 2022] PUT /identity/v3/projects/790a398668664cdb83089761d62a0cc6/users/8cfd60845f984b0591ac03c53003a06b/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:22.683589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.683881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.684271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.715565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] MFA Rules not processed for user `8cfd60845f984b0591ac03c53003a06b`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:22.717589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:22.790503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 919/1835] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:22.795374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.864154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e7f39-df4d-4cd5-a055-a27d2bf9dd7c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.869189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:22.869966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:22.870392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:22.870729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:22.871518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:22.873734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:22.914162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 917/1836] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:22 2022] POST /identity/v3/projects => generated 407 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:22.918397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:22.941152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da9227-a4d7-499b-bbf7-9321794ceb02 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:22.959580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:23.007689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.008433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.008742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.009040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.009773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.011732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.034694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 920/1837] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:22 2022] POST /identity/v3/users => generated 310 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:23.038664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:23.049775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd346c2-4c57-4517-a0c6-b3d606320103 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:23.054362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.055037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.055316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.055587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.056325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.057967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.079966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 918/1838] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:23.083866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:23.095973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59096fa1-1e2b-4d07-ba5b-60a5ca420de0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:23.101161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.102246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.102536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.102818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] PATH_INFO: `/v3/projects/31dcc19c10e8454f874717f7bd974417/users/22e3df884b8847dcb70b57f82376f5ec/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.124677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31dcc19c10e8454f874717f7bd974417, user_id=22e3df884b8847dcb70b57f82376f5ec, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.127314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.144494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 921/1839] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:23 2022] PUT /identity/v3/projects/31dcc19c10e8454f874717f7bd974417/users/22e3df884b8847dcb70b57f82376f5ec/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:23.149168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:23.161210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c35e97-43f9-4513-b894-07d48a382237 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:23.166609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:23.253479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efab49-fa47-49ea-a452-8aced276daf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.254264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efab49-fa47-49ea-a452-8aced276daf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.254493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efab49-fa47-49ea-a452-8aced276daf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.254963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efab49-fa47-49ea-a452-8aced276daf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.255736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efab49-fa47-49ea-a452-8aced276daf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.257469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efab49-fa47-49ea-a452-8aced276daf8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.269162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 919/1840] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:23.273894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:23.289243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5185f-78bf-4ff0-83c4-4930d1f7aac7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:23.296835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.297872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.298669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.299016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] PATH_INFO: `/v3/projects/31dcc19c10e8454f874717f7bd974417/users/22e3df884b8847dcb70b57f82376f5ec/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.301917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=31dcc19c10e8454f874717f7bd974417, user_id=22e3df884b8847dcb70b57f82376f5ec, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.304862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.317262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 922/1841] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:23 2022] PUT /identity/v3/projects/31dcc19c10e8454f874717f7bd974417/users/22e3df884b8847dcb70b57f82376f5ec/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:23.328245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.328532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.328859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.358363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] MFA Rules not processed for user `22e3df884b8847dcb70b57f82376f5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:23.359589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:23.442862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 920/1842] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:23.456363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:23.458299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:23.579331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd853d-a73c-4a13-bf8a-c741f4bb42fa admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:23.583179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.584003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.584288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.584561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.595917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.597701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.609003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 923/1843] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:23 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:23.841851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:23.843321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:23.857803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749a4b4-3865-4b1f-bf79-05362a27acd8 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:23.874613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fabb7-f380-474b-ad60-56a96208ffe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:23.875881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fabb7-f380-474b-ad60-56a96208ffe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:23.876398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fabb7-f380-474b-ad60-56a96208ffe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:23.876952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fabb7-f380-474b-ad60-56a96208ffe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:23.969956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fabb7-f380-474b-ad60-56a96208ffe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:23.971429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fabb7-f380-474b-ad60-56a96208ffe0 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:23.983600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 921/1844] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:25.285103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:25.287322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:25.358236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a180efa-a7b1-455d-bcd2-c19b5e9e9f52 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:25.363162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:25.364058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:25.364337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:25.364614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] PATH_INFO: `/v3/users/d56538ea1461447dbe2134403ff3e84f` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:25.365855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56538ea1461447dbe2134403ff3e84f)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:25.368078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:25.385443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:25.421705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56538ea1461447dbe2134403ff3e84f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:25.422401 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56538ea1461447dbe2134403ff3e84f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:25.436138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56538ea1461447dbe2134403ff3e84f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:25.436824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56538ea1461447dbe2134403ff3e84f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:25.445763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56538ea1461447dbe2134403ff3e84f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:25.458606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 924/1845] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:25 2022] DELETE /identity/v3/users/d56538ea1461447dbe2134403ff3e84f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:25.463376 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:25.463935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:25.463935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:25.482949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:25.484172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:25.514948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:25.551823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 922/1846] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:25.562159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:25.563756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:25.628680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6e1bf-5467-4733-9dcf-d59906f31d3e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:25.632549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:25.633244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:25.633526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:25.633801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:25.643966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:25.645453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:25.655038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 925/1847] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:26.019875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:26.021558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:26.034279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e16fb0-8122-4720-a803-3f62f1ee70e6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:26.044536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:26.045478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:26.045756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:26.046029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] PATH_INFO: `/v3/projects/9635d9cc72f742b49d104efd71775778` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:26.047280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9635d9cc72f742b49d104efd71775778)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:26.049610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:26.113164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9635d9cc72f742b49d104efd71775778', 'request_id': 'req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.123974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9635d9cc72f742b49d104efd71775778', 'request_id': 'req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.132577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9635d9cc72f742b49d104efd71775778', 'request_id': 'req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.147392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] The token cache is being invalidate because project 9635d9cc72f742b49d104efd71775778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:26.147676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.148365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.150642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 923/1848] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:26 2022] DELETE /identity/v3/projects/9635d9cc72f742b49d104efd71775778 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:26.154937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:26.165572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00e00-1e81-4511-8b1b-e49e276d205d service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:26.224728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:26.225540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:26.225823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:26.226101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] PATH_INFO: `/v3/users/9bfb22266687427eaaed000d5f86aa57` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:26.227306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfb22266687427eaaed000d5f86aa57)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:26.229416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:26.275679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:26.283715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfb22266687427eaaed000d5f86aa57'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.284561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfb22266687427eaaed000d5f86aa57'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.297282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfb22266687427eaaed000d5f86aa57'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.298283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfb22266687427eaaed000d5f86aa57'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.305868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfb22266687427eaaed000d5f86aa57'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.318761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 926/1849] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:26 2022] DELETE /identity/v3/users/9bfb22266687427eaaed000d5f86aa57 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:26.603144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:26.604826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:26.623787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71d30-f9ef-4536-9bb6-cbc9e16872c4 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:26.695447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:26.696390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:26.696699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:26.696978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] PATH_INFO: `/v3/projects/fa4024bf94444268a86966726891f09d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:26.698218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4024bf94444268a86966726891f09d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:26.700166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:26.747241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4024bf94444268a86966726891f09d', 'request_id': 'req-ac1931da-3950-44bf-8078-ea35b71c2929'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.751536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:26.756101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4024bf94444268a86966726891f09d', 'request_id': 'req-ac1931da-3950-44bf-8078-ea35b71c2929'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.763548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4024bf94444268a86966726891f09d', 'request_id': 'req-ac1931da-3950-44bf-8078-ea35b71c2929'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.775750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] The token cache is being invalidate because project fa4024bf94444268a86966726891f09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:26.776038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.776725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac1931da-3950-44bf-8078-ea35b71c2929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:26.778301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 924/1850] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:26 2022] DELETE /identity/v3/projects/fa4024bf94444268a86966726891f09d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:26.782749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:26.784375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:26.804962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc005eb0-1aaf-4a52-ac30-32e8ce147a17 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:26.993125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:26.993933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:26.994206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:26.994474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] PATH_INFO: `/v3/users/d6688226568c45d09efebd2fb03e3a3f` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:26.995682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6688226568c45d09efebd2fb03e3a3f)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:26.997802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:27.058743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6688226568c45d09efebd2fb03e3a3f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.059479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6688226568c45d09efebd2fb03e3a3f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.072756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6688226568c45d09efebd2fb03e3a3f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.073453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6688226568c45d09efebd2fb03e3a3f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.082232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6688226568c45d09efebd2fb03e3a3f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.094795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 927/1851] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:26 2022] DELETE /identity/v3/users/d6688226568c45d09efebd2fb03e3a3f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:27.378837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f71fb-8c49-4685-a081-d2cca547b9be None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.379121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f71fb-8c49-4685-a081-d2cca547b9be None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.379401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f71fb-8c49-4685-a081-d2cca547b9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.396476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:27.397933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.415585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-501f71fb-8c49-4685-a081-d2cca547b9be None None] MFA Rules not processed for user `94b91e199622457787fa2a7e78d743c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:27.419414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-501f71fb-8c49-4685-a081-d2cca547b9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.453251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:27.490023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3f324-5522-4cdf-b734-9fb4f9e0e2d4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:27.494963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:27.496055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.496055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.496343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] PATH_INFO: `/v3/projects/e91d6cdcf166475089d246b736f2f852` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.496924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 925/1852] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 30 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:27.498018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91d6cdcf166475089d246b736f2f852)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:27.500153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:27.552953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91d6cdcf166475089d246b736f2f852', 'request_id': 'req-58ee1856-2775-433a-ab80-7b6b9562d591'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.560692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91d6cdcf166475089d246b736f2f852', 'request_id': 'req-58ee1856-2775-433a-ab80-7b6b9562d591'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.566734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91d6cdcf166475089d246b736f2f852', 'request_id': 'req-58ee1856-2775-433a-ab80-7b6b9562d591'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.582086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] The token cache is being invalidate because project e91d6cdcf166475089d246b736f2f852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:27.582377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.583103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:27.585013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 928/1853] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:27 2022] DELETE /identity/v3/projects/e91d6cdcf166475089d246b736f2f852 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:27.591853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3067e-7245-453c-9c00-b997c90cc8de None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.592145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3067e-7245-453c-9c00-b997c90cc8de None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.592471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3067e-7245-453c-9c00-b997c90cc8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.614502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-fcb3067e-7245-453c-9c00-b997c90cc8de None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:27.615890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fcb3067e-7245-453c-9c00-b997c90cc8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.671012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:27.673105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.694159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 926/1854] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:27.703642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.704084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.704698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.734940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:27.736234 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.749593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 927/1855] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:27.750440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:27.752124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.767151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee1856-2775-433a-ab80-7b6b9562d591 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:27.771760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83968f4-c94d-49cf-aa1c-35c6ec0764c7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:27.772213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:27.773262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.773668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.774007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.776235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:27.778776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:27.779658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.779944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.780270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.787485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:27.789094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:27.790952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:27.792931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:27.798256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 929/1856] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:27.798897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:27.812653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 928/1857] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:27.816013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ae9b7-7561-4245-b268-69c3bde68d93 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:27.824423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:27.825292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.825697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.826099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.827663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:27.828034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:27.829932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:27.846229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 930/1858] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:27.850902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:27.869436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b324f8-a553-475f-849d-ceeb8949b552 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:27.875061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:27.876134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:27.876252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:27.876443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:27.877238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:27.879595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:27.916353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 929/1859] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:27 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:27.921700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:27.939300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b650d4-7816-4abb-801c-8647cbe737ce admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:28.029933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:28.054946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:28.058155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.058443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.058725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.061097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:28.066641 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:28.118029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 931/1860] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:27 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:28.123736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:28.138930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8408142-4992-4d18-8884-84c90650bb44 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:28.158262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:28.159072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.159491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.159765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.160580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:28.162302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:28.181002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 930/1861] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:28.185913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:28.210090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1decc-c246-4be3-8ebb-ab69b391c445 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:28.230322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e210-a0b0-48b5-957f-90b2f9697144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:28.231239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e210-a0b0-48b5-957f-90b2f9697144 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.233892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e210-a0b0-48b5-957f-90b2f9697144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.234236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e210-a0b0-48b5-957f-90b2f9697144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.235010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5e210-a0b0-48b5-957f-90b2f9697144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:28.248823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5e210-a0b0-48b5-957f-90b2f9697144 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:28.284906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 932/1862] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:28.289308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:28.305695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ec5d-a6ec-43ed-aadd-48ed12252ec3 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:28.327190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:28.328228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.328513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.328814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] PATH_INFO: `/v3/projects/3a351b3c82fb4d7aa582989994244d6b/users/474dcadd25d74e25b8819af0adf0fa25/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.381206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a351b3c82fb4d7aa582989994244d6b, user_id=474dcadd25d74e25b8819af0adf0fa25, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:28.385525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:28.417024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 931/1863] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:28 2022] PUT /identity/v3/projects/3a351b3c82fb4d7aa582989994244d6b/users/474dcadd25d74e25b8819af0adf0fa25/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:28.429565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.430102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.430102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.463880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] MFA Rules not processed for user `474dcadd25d74e25b8819af0adf0fa25`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:28.465249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:28.564949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 933/1864] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:28.567182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:28.602471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:28.686246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7462-e359-47df-bdda-b7b4deb5df51 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:28.696188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:28.697152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.697548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.698772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.699573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:28.701707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:28.741441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 932/1865] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:28 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:28.745149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:28.758941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3acf0d-b6df-4fba-8260-9b43fd586892 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:28.913478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:28.914905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:28.915975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:28.916331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:28.917835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:28.925299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:28.975771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 934/1866] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:28 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:28.980034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:29.006275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa3b6b-4695-46ae-9c03-4f1957fbb267 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:29.036059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:29.037138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.037802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.038270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.042019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:29.045208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:29.064053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 933/1867] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:29.068998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:29.097961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809a6d3-b6b4-477f-a9f7-4aa3b5a3e673 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:29.108426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f863e-6750-4742-af48-3557d6d37aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:29.112046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f863e-6750-4742-af48-3557d6d37aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.114036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f863e-6750-4742-af48-3557d6d37aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.114496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f863e-6750-4742-af48-3557d6d37aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.116322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f863e-6750-4742-af48-3557d6d37aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:29.119781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f863e-6750-4742-af48-3557d6d37aa3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:29.141349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 935/1868] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:29.146273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:29.162577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa3d6b-be0b-4bbb-9d01-ee3528f2d5a2 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:29.178928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:29.179958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.180347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.180838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] PATH_INFO: `/v3/projects/cd5e481afcbb47ee95217ac3aa3da092/users/5812b61a6efc47b9a671d50782da5e50/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.229929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5e481afcbb47ee95217ac3aa3da092, user_id=5812b61a6efc47b9a671d50782da5e50, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:29.238082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:29.279820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 934/1869] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:29 2022] PUT /identity/v3/projects/cd5e481afcbb47ee95217ac3aa3da092/users/5812b61a6efc47b9a671d50782da5e50/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:29.292053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.292575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.292883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.401461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] MFA Rules not processed for user `5812b61a6efc47b9a671d50782da5e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:29.403412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:29.487512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:29.571046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 936/1870] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:29.575673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:29.699438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c724e-f2b5-47fd-b12f-529491dbc0d8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:29.729871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:29.731759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.732204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.733162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.734236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:29.736749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:29.803353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:29.804588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 935/1871] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:29 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:29.827169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2c250-c9dc-4042-9775-1e5a0e65bb00 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:29.916619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb707962-46b0-4594-9227-8a1d4258a986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:29.917360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707962-46b0-4594-9227-8a1d4258a986 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.917647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707962-46b0-4594-9227-8a1d4258a986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.917920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707962-46b0-4594-9227-8a1d4258a986 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.919539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707962-46b0-4594-9227-8a1d4258a986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:29.921010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707962-46b0-4594-9227-8a1d4258a986 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:29.964465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:29.983265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 937/1872] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:29 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:29.989937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd8587-a225-43f7-8945-5e20c4cec031 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:29.995272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97cbf-cce8-46b0-978d-c1313ccc89c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:29.995977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97cbf-cce8-46b0-978d-c1313ccc89c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:29.996207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97cbf-cce8-46b0-978d-c1313ccc89c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:29.996486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97cbf-cce8-46b0-978d-c1313ccc89c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:29.998231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97cbf-cce8-46b0-978d-c1313ccc89c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.000387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97cbf-cce8-46b0-978d-c1313ccc89c3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.030036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6970b1c-ced9-4fa0-bd80-ce5985a0ab16 None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.030142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6970b1c-ced9-4fa0-bd80-ce5985a0ab16 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.033185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6970b1c-ced9-4fa0-bd80-ce5985a0ab16 None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.040587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 938/1873] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:33:30 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:33:30.041926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 936/1874] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:30.046971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6970b1c-ced9-4fa0-bd80-ce5985a0ab16 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.052932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b30728-8d9e-4337-942d-232411d6dd9e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.053226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b30728-8d9e-4337-942d-232411d6dd9e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.053496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b30728-8d9e-4337-942d-232411d6dd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.074973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6970b1c-ced9-4fa0-bd80-ce5985a0ab16 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.080600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.081714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.082086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.082872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] PATH_INFO: `/v3/projects/07b337c80e024054a44156e1081365f0/users/cb88c9e9a747416eba9979c6d57ae8b3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.086820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-26b30728-8d9e-4337-942d-232411d6dd9e None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:30.088723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-26b30728-8d9e-4337-942d-232411d6dd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.101687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b337c80e024054a44156e1081365f0, user_id=cb88c9e9a747416eba9979c6d57ae8b3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.105060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.140350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 939/1875] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:30 2022] PUT /identity/v3/projects/07b337c80e024054a44156e1081365f0/users/cb88c9e9a747416eba9979c6d57ae8b3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:30.143611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.150137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:30.226368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 937/1876] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:30.230491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36641-9b08-4408-98bb-416d8cee7dc7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.230780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36641-9b08-4408-98bb-416d8cee7dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.231087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36641-9b08-4408-98bb-416d8cee7dc7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.237023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 938/1877] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:33:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:33:30.248787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36641-9b08-4408-98bb-416d8cee7dc7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.251355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73d572-b634-4dac-ae6d-e90c9cf65021 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.252264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-99d36641-9b08-4408-98bb-416d8cee7dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.256594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e37921-a6b3-4f40-98ba-f3893d3fad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.257412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e37921-a6b3-4f40-98ba-f3893d3fad14 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.257698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e37921-a6b3-4f40-98ba-f3893d3fad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.257971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e37921-a6b3-4f40-98ba-f3893d3fad14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.259549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e37921-a6b3-4f40-98ba-f3893d3fad14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.261677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e37921-a6b3-4f40-98ba-f3893d3fad14 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.294025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 940/1878] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:30.295720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.296010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.296285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.328531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:30.330800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.384380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:30.395861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36641-9b08-4408-98bb-416d8cee7dc7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.404256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82967440-575c-49e7-8a22-30be44402950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.404924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82967440-575c-49e7-8a22-30be44402950 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.405210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82967440-575c-49e7-8a22-30be44402950 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.405490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82967440-575c-49e7-8a22-30be44402950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.415293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82967440-575c-49e7-8a22-30be44402950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.416988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82967440-575c-49e7-8a22-30be44402950 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.429142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 939/1879] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:30.437644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82967440-575c-49e7-8a22-30be44402950 service nova] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.439064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 941/1880] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:30.448478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82967440-575c-49e7-8a22-30be44402950 service nova] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.452410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.454246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.454631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20aaa-c02a-4212-a253-1108640bc327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.455574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20aaa-c02a-4212-a253-1108640bc327 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.455854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20aaa-c02a-4212-a253-1108640bc327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.456125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20aaa-c02a-4212-a253-1108640bc327 admin admin] PATH_INFO: `/v3/projects/07b337c80e024054a44156e1081365f0/users/cb88c9e9a747416eba9979c6d57ae8b3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.458147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc20aaa-c02a-4212-a253-1108640bc327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b337c80e024054a44156e1081365f0, user_id=cb88c9e9a747416eba9979c6d57ae8b3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.460831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc20aaa-c02a-4212-a253-1108640bc327 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.474067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 940/1881] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:30 2022] PUT /identity/v3/projects/07b337c80e024054a44156e1081365f0/users/cb88c9e9a747416eba9979c6d57ae8b3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:30.483550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd85d97-c271-4989-91db-165ba44f5b90 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.484370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.484707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.484992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.518122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] MFA Rules not processed for user `cb88c9e9a747416eba9979c6d57ae8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:30.519356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.551888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.552546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.552783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.553058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.561912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.563519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.573765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 942/1882] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:30.587299 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 941/1883] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:30.599260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.600686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.665014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b3667-d402-489c-8468-e176e1838e87 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.670057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.671077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.671533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.671872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.682186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.683637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.694651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 943/1884] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:30.744501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:30.746379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:30.760657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99978952-4cb3-4acd-ab0e-da047c9f094a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:30.765182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c541dff-32f6-4f19-a2a4-82ee98362d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:30.765854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541dff-32f6-4f19-a2a4-82ee98362d25 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:30.766132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541dff-32f6-4f19-a2a4-82ee98362d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:30.766412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541dff-32f6-4f19-a2a4-82ee98362d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:30.797257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:30.838272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c541dff-32f6-4f19-a2a4-82ee98362d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:30.839975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c541dff-32f6-4f19-a2a4-82ee98362d25 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:30.850506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 942/1885] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:36.599518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:36.601177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:36.665797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:36.679515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a3ba9-c5ce-4395-bffc-26fe3e5f9a7f service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:36.684449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:36.685672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:36.686015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:36.686450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] PATH_INFO: `/v3/users/5ee8513f64a644739709210f7803e031` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:36.687661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee8513f64a644739709210f7803e031)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:36.689629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:36.739620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee8513f64a644739709210f7803e031'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:36.740971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee8513f64a644739709210f7803e031'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:36.754257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee8513f64a644739709210f7803e031'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:36.755197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee8513f64a644739709210f7803e031'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:36.765747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee8513f64a644739709210f7803e031'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:36.787719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 944/1886] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:36 2022] DELETE /identity/v3/users/5ee8513f64a644739709210f7803e031 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:36.792311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:36.792844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:36.792927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:36.811351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:36.812560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:36.871752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 943/1887] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:36.881771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:36.883364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:36.951923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fba2f2-faa6-4ee9-bed6-9167178cfb0f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:36.956015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979ac56-2b46-4cd0-b0f3-e9494cd471dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:36.956658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979ac56-2b46-4cd0-b0f3-e9494cd471dd service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:36.957005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979ac56-2b46-4cd0-b0f3-e9494cd471dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:36.957282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979ac56-2b46-4cd0-b0f3-e9494cd471dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:36.967153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979ac56-2b46-4cd0-b0f3-e9494cd471dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:36.968469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979ac56-2b46-4cd0-b0f3-e9494cd471dd service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:36.978602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 945/1888] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:37.103272 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:37.105178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:37.117753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2748a-e037-4752-ba06-d2e3c46ee75e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:37.122527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:37.123631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:37.123976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:37.124311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] PATH_INFO: `/v3/users/474dcadd25d74e25b8819af0adf0fa25` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:37.125909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] RBAC: Authorizing `identity:delete_user(user_id=474dcadd25d74e25b8819af0adf0fa25)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:37.127882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:37.198886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474dcadd25d74e25b8819af0adf0fa25'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.199543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474dcadd25d74e25b8819af0adf0fa25'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.213788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474dcadd25d74e25b8819af0adf0fa25'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.214606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474dcadd25d74e25b8819af0adf0fa25'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.223253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474dcadd25d74e25b8819af0adf0fa25'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.236393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 944/1889] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:37 2022] DELETE /identity/v3/users/474dcadd25d74e25b8819af0adf0fa25 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:37.241017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:37.241303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:37.241576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:37.258931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:37.260178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:37.274707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:37.276139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:37.326548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 946/1890] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:37.334178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:37.336914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:37.338447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:37.357087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:37.358266 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1242-d629-4282-b7ae-ada411c6602a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:37.363592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:37.364516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:37.364864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:37.365170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] PATH_INFO: `/v3/projects/9a9cceddd2ff4689bf553a6166a78b1a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:37.366379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9cceddd2ff4689bf553a6166a78b1a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:37.368341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:37.401529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dc813-5da4-4ec5-bc03-ab2c868e44bd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:37.405816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:37.406521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:37.406918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:37.407216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:37.416802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9cceddd2ff4689bf553a6166a78b1a', 'request_id': 'req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.416988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:37.418476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:37.425207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9cceddd2ff4689bf553a6166a78b1a', 'request_id': 'req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.430651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 947/1891] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:37.434213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9cceddd2ff4689bf553a6166a78b1a', 'request_id': 'req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.448398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] The token cache is being invalidate because project 9a9cceddd2ff4689bf553a6166a78b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:37.448821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.449710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.454842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 945/1892] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:37 2022] DELETE /identity/v3/projects/9a9cceddd2ff4689bf553a6166a78b1a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:37.456084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:37.466772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636089b9-beae-4b3b-8cbb-3952a77d2264 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:37.531064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:37.532064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:37.532226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:37.532501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] PATH_INFO: `/v3/users/95efa5fc30f7416cabf31a40c2a553e3` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:37.533833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95efa5fc30f7416cabf31a40c2a553e3)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:37.535689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:37.584782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95efa5fc30f7416cabf31a40c2a553e3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.585456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95efa5fc30f7416cabf31a40c2a553e3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.597908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95efa5fc30f7416cabf31a40c2a553e3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.598678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95efa5fc30f7416cabf31a40c2a553e3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.608026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95efa5fc30f7416cabf31a40c2a553e3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:37.622788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 948/1893] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:37 2022] DELETE /identity/v3/users/95efa5fc30f7416cabf31a40c2a553e3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:37.825852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:37.827571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:37.846816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a10ce-8f63-4d05-a1b9-1d5b594f99a8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:37.942946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:37.944029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:37.944392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:37.944762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] PATH_INFO: `/v3/projects/3a351b3c82fb4d7aa582989994244d6b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:37.946822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a351b3c82fb4d7aa582989994244d6b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:37.949180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:37.970703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:37.972134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.004042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d42-e5b3-4e47-9bfc-ba0b624701f9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.025910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a351b3c82fb4d7aa582989994244d6b', 'request_id': 'req-1709e9f7-441a-4d46-a083-18fc5f1baee8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.038073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a351b3c82fb4d7aa582989994244d6b', 'request_id': 'req-1709e9f7-441a-4d46-a083-18fc5f1baee8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.049700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a351b3c82fb4d7aa582989994244d6b', 'request_id': 'req-1709e9f7-441a-4d46-a083-18fc5f1baee8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.053737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:38.060988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:38.065609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] The token cache is being invalidate because project 3a351b3c82fb4d7aa582989994244d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:38.065927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.067200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.070363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 946/1894] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:37 2022] DELETE /identity/v3/projects/3a351b3c82fb4d7aa582989994244d6b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:38.074346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.076109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.097675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.098894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.099176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.099453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] PATH_INFO: `/v3/projects/90255a9629f54b43ab3f6850012ca071` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.100676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90255a9629f54b43ab3f6850012ca071)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.102639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.160750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709e9f7-441a-4d46-a083-18fc5f1baee8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.175028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90255a9629f54b43ab3f6850012ca071', 'request_id': 'req-edf2b365-88f7-4886-81ac-28b18d18f195'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.182434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90255a9629f54b43ab3f6850012ca071', 'request_id': 'req-edf2b365-88f7-4886-81ac-28b18d18f195'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.188268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90255a9629f54b43ab3f6850012ca071', 'request_id': 'req-edf2b365-88f7-4886-81ac-28b18d18f195'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.200914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] The token cache is being invalidate because project 90255a9629f54b43ab3f6850012ca071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:38.201125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.201751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-edf2b365-88f7-4886-81ac-28b18d18f195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.203315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 949/1895] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:37 2022] DELETE /identity/v3/projects/90255a9629f54b43ab3f6850012ca071 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:38.208523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23fada-6e5d-4775-a64d-3e14ba8edc89 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.208849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23fada-6e5d-4775-a64d-3e14ba8edc89 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.209117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23fada-6e5d-4775-a64d-3e14ba8edc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.227971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2e23fada-6e5d-4775-a64d-3e14ba8edc89 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:38.229229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2e23fada-6e5d-4775-a64d-3e14ba8edc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.232781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.233683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.234019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.234278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] PATH_INFO: `/v3/users/5812b61a6efc47b9a671d50782da5e50` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.235507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5812b61a6efc47b9a671d50782da5e50)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.237501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.238478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 950/1896] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:38.246029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.246310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.246610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.265219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:38.266707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.275593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 951/1897] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:38.280359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.286499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5812b61a6efc47b9a671d50782da5e50'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.287350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5812b61a6efc47b9a671d50782da5e50'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.289792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc517f16-80d7-4997-8eb0-86fff12d715c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.294840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.295500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.295780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.296103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.297008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:38.297334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.299304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.300033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5812b61a6efc47b9a671d50782da5e50'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.300789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5812b61a6efc47b9a671d50782da5e50'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.309686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5812b61a6efc47b9a671d50782da5e50'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.315564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 952/1898] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:33:38.316803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.318495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.322351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 947/1899] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:38 2022] DELETE /identity/v3/users/5812b61a6efc47b9a671d50782da5e50 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:38.337981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7d8fa-0f6f-41ce-9ed6-37bd2dad6cbe admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.398488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.399191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.399476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.399749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.400451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.402468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.430122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 953/1900] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:38.434555 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.445432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50718273-b1a8-4dd8-ba38-d3abfa78a837 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.470955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:38.508243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.508971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.509253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.509531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.510233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.512165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.536101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 948/1901] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:38.537168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.549152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c53e-930b-44bd-a9e7-a6aed10c3920 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.554902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.555684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.555956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.556265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.557206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.558812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.570984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 954/1902] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:38.575488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.586794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600905f-61f4-4521-98ab-30a0fb5aeff9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.592329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.593112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.593391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.593667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.594590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.596717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.605298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.607622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.609550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 949/1903] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:38.613855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.621261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93937c4e-2810-4cd7-bfa1-e90848242a7a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.625472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8926d-5872-4f2b-9f40-789e03814963 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.627903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.628941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.629222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.629648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] PATH_INFO: `/v3/projects/cd5e481afcbb47ee95217ac3aa3da092` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.630631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965198ef-9dd6-46a4-8375-53f2b585dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.630862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5e481afcbb47ee95217ac3aa3da092)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.631537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965198ef-9dd6-46a4-8375-53f2b585dfc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.631815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965198ef-9dd6-46a4-8375-53f2b585dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.632098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965198ef-9dd6-46a4-8375-53f2b585dfc0 admin admin] PATH_INFO: `/v3/projects/55a304c67f47459bba1c5de24ea44f1c/users/a8e8772662c14a2d9cfa8a5f39bd0ece/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.632989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.645302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:38.651438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965198ef-9dd6-46a4-8375-53f2b585dfc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a304c67f47459bba1c5de24ea44f1c, user_id=a8e8772662c14a2d9cfa8a5f39bd0ece, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.654224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965198ef-9dd6-46a4-8375-53f2b585dfc0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.674274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 950/1904] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:38 2022] PUT /identity/v3/projects/55a304c67f47459bba1c5de24ea44f1c/users/a8e8772662c14a2d9cfa8a5f39bd0ece/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:38.686217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.686595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.687062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.698333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5e481afcbb47ee95217ac3aa3da092', 'request_id': 'req-f00b960b-c12f-4c1d-a97c-0e6c45178d81'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.707453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5e481afcbb47ee95217ac3aa3da092', 'request_id': 'req-f00b960b-c12f-4c1d-a97c-0e6c45178d81'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.714041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5e481afcbb47ee95217ac3aa3da092', 'request_id': 'req-f00b960b-c12f-4c1d-a97c-0e6c45178d81'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.717219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] MFA Rules not processed for user `a8e8772662c14a2d9cfa8a5f39bd0ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:38.718572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.727129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] The token cache is being invalidate because project cd5e481afcbb47ee95217ac3aa3da092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:38.727416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.728053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.730022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 955/1905] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:38 2022] DELETE /identity/v3/projects/cd5e481afcbb47ee95217ac3aa3da092 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:38.734185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.735361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.792881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 951/1906] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:38.798028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.799445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:38.813985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b960b-c12f-4c1d-a97c-0e6c45178d81 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.816328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df68c19-4914-48e7-a446-b0008a1a733f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:38.820889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.822344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.822344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.822530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:38.822635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] PATH_INFO: `/v3/users/cb88c9e9a747416eba9979c6d57ae8b3` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.823265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:38.823566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb88c9e9a747416eba9979c6d57ae8b3)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.823680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:38.823976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:38.824932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:38.825647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.827130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:38.860268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 952/1907] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:38.864739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:38.888185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb88c9e9a747416eba9979c6d57ae8b3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.888896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb88c9e9a747416eba9979c6d57ae8b3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.904547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb88c9e9a747416eba9979c6d57ae8b3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.905298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb88c9e9a747416eba9979c6d57ae8b3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.913674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb88c9e9a747416eba9979c6d57ae8b3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:38.914920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:38.927468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 956/1908] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:38 2022] DELETE /identity/v3/users/cb88c9e9a747416eba9979c6d57ae8b3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:38.930558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39851216-63ef-4e2d-b1f5-6fb5557c72f0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.008063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.008744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.009022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.009293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.009994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.011993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.033521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 953/1909] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:38 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:39.037896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.039326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.053047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fb3e5-db13-41e0-8b4e-6aa7e25a5503 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.058044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.058719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.059037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.059254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.060091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.061720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.072227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 957/1910] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:39.076515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.087557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be52-ce0a-4856-ac71-b7cb237b51f9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.105946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.107245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.107245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.107245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] PATH_INFO: `/v3/projects/3d45ba62e84f43dfb0e9083a793a1112/users/ed6695c134ba42a2bb8e5d1f0d8c98a8/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.136434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d45ba62e84f43dfb0e9083a793a1112, user_id=ed6695c134ba42a2bb8e5d1f0d8c98a8, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.140332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.162903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 954/1911] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:39 2022] PUT /identity/v3/projects/3d45ba62e84f43dfb0e9083a793a1112/users/ed6695c134ba42a2bb8e5d1f0d8c98a8/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:39.167589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.189961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86965d-aa2e-47b6-b74a-b7e061bd95f2 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.196260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:39.310503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22aca0-4cca-4273-91a1-00c7127086a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.311305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22aca0-4cca-4273-91a1-00c7127086a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.311733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22aca0-4cca-4273-91a1-00c7127086a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.312006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22aca0-4cca-4273-91a1-00c7127086a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.313005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22aca0-4cca-4273-91a1-00c7127086a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.315766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22aca0-4cca-4273-91a1-00c7127086a2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.329998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 958/1912] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:39.334742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.351062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e9908-5b82-4eae-a21a-6168371810dd admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.357490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.358590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.358890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.359125 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] PATH_INFO: `/v3/projects/3d45ba62e84f43dfb0e9083a793a1112/users/ed6695c134ba42a2bb8e5d1f0d8c98a8/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.361519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d45ba62e84f43dfb0e9083a793a1112, user_id=ed6695c134ba42a2bb8e5d1f0d8c98a8, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.364444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.393537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.393897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.394098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.395946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 955/1913] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:39 2022] PUT /identity/v3/projects/3d45ba62e84f43dfb0e9083a793a1112/users/ed6695c134ba42a2bb8e5d1f0d8c98a8/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:39.434077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] MFA Rules not processed for user `ed6695c134ba42a2bb8e5d1f0d8c98a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:39.435883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.562715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 959/1914] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:39.579413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.581715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.618421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.619954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.669579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dbb3a-0099-45b6-8511-9cba31be5d3d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.674876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.675627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.675918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.676183 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.687014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.688710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.703968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 956/1915] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:39.705304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60373ee-e25d-48ff-a45f-d4102b638b6c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.710260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.711491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.711924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.712356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] PATH_INFO: `/v3/projects/07b337c80e024054a44156e1081365f0` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.713787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b337c80e024054a44156e1081365f0)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.715831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.760243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:39.776214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b337c80e024054a44156e1081365f0', 'request_id': 'req-a421a7b3-a59c-4452-8f37-315ef2733169'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:39.783324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b337c80e024054a44156e1081365f0', 'request_id': 'req-a421a7b3-a59c-4452-8f37-315ef2733169'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:39.786775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.790317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b337c80e024054a44156e1081365f0', 'request_id': 'req-a421a7b3-a59c-4452-8f37-315ef2733169'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:39.796257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611cfa3-ee4d-45d1-adaf-c8d37c39fe81 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.805565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] The token cache is being invalidate because project 07b337c80e024054a44156e1081365f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:39.805565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:39.806388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a421a7b3-a59c-4452-8f37-315ef2733169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:39.808337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 960/1916] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:39 2022] DELETE /identity/v3/projects/07b337c80e024054a44156e1081365f0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:39.814252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c289bb-9403-429b-9e05-8bf1d58e8d7b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.814534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c289bb-9403-429b-9e05-8bf1d58e8d7b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.814807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c289bb-9403-429b-9e05-8bf1d58e8d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.834209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-57c289bb-9403-429b-9e05-8bf1d58e8d7b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:39.835842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-57c289bb-9403-429b-9e05-8bf1d58e8d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.849745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.850439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.850685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.850983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.852879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.858391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:39.920501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 961/1917] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:39.931829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:39.932286 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.932580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.933317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:39.933504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.943191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 957/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:39.956478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:39.957747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:39.971633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 962/1919] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:39.973013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:39.986047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8a7ef-1ea5-460b-8c8e-bf9763bf557d service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:39.995212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:39.997703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:39.998144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:39.998471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:39.999216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:39.999529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.001815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.023827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 958/1920] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:40.028894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.045666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf8d36-359e-498b-9fb1-411ca39a77c9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.057819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.058443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.058725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.058999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.059802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.064924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.147812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.155163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 963/1921] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:40 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:40.168685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd2091-2fa4-447d-8b12-54a36f96f0e5 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.238592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.239296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.239692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.239969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.240731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.242740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.273324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 959/1922] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:40 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:40.276351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.289254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bda0c-b83e-496e-985b-4a6a2d5aaf46 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.294124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.294809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.295085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.295356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.296124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.298140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.313694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 964/1923] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:40.318311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.331874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df63f8-ede9-493b-bf06-c6a7a5a5bced admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.339350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc14935-78ad-4c3f-b029-ac97dedf72d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.340604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc14935-78ad-4c3f-b029-ac97dedf72d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.341221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc14935-78ad-4c3f-b029-ac97dedf72d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.341759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc14935-78ad-4c3f-b029-ac97dedf72d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.342827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc14935-78ad-4c3f-b029-ac97dedf72d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.344603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc14935-78ad-4c3f-b029-ac97dedf72d5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.357916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 960/1924] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:40.362908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.376606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d0bfa-4e2c-4fcc-b531-aad09c218df5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.387724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.388684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.388978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.389356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] PATH_INFO: `/v3/projects/367c3bb8852f4ddeb87eaa809c8c1fd5/users/a64f75f4402740308148fd7e33c0ccf8/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.406766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] RBAC: Authorizing `identity:create_grant(project_id=367c3bb8852f4ddeb87eaa809c8c1fd5, user_id=a64f75f4402740308148fd7e33c0ccf8, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.409655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.426748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 965/1925] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:40 2022] PUT /identity/v3/projects/367c3bb8852f4ddeb87eaa809c8c1fd5/users/a64f75f4402740308148fd7e33c0ccf8/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:40.438879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.439062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.439405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.473500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] MFA Rules not processed for user `a64f75f4402740308148fd7e33c0ccf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:40.474835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:40.577688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:40.585146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 961/1926] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:40.589203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.603546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.605472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:40.672001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:40.679445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f2095-194b-48b7-a216-7bc1e1b1f55e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.687414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.689079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.690155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.691094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.691764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.695023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.712988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcef11-5032-4e7e-8aa8-39c55ebc3205 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.719706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.720852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.721141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.721418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] PATH_INFO: `/v3/users/8cfd60845f984b0591ac03c53003a06b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.722983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cfd60845f984b0591ac03c53003a06b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.725357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.749588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 966/1927] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:40 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:40.751895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.796837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cfd60845f984b0591ac03c53003a06b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:40.797876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cfd60845f984b0591ac03c53003a06b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:40.811037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cfd60845f984b0591ac03c53003a06b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:40.811762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cfd60845f984b0591ac03c53003a06b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:40.819975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a34c2ca8-51a1-4e56-9c68-f2c640d395c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cfd60845f984b0591ac03c53003a06b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:40.836343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 962/1928] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:40 2022] DELETE /identity/v3/users/8cfd60845f984b0591ac03c53003a06b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:40.838464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.839163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.839163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.842979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28d621-c0bb-4879-a74d-e87db832d0d2 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.858025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:40.859292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:40.901628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.902409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.902760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.903102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.903898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.906007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.921587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 963/1929] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:40.932179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.932801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 967/1930] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:40 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:40.934087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.934853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:40.935698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:40.947327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd895-b88b-4eed-9c9d-43b542d1cd14 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.952778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.953461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.953740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.954010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:40.954737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:40.956351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:40.960483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:40.969634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 968/1931] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:40.974129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:40.986798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68d38-aaa3-4e08-aa83-000edbe3d89c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:40.992075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:40.993015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:40.993295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:40.993583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] PATH_INFO: `/v3/projects/c1574d38a0f847a497811efa97a6681f/users/ac66eb0e72784a8b948b63108f201da3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.003165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377896b-102e-4540-ba97-82783669c1d1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:41.007290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:41.008300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.008587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.009162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.011610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1574d38a0f847a497811efa97a6681f, user_id=ac66eb0e72784a8b948b63108f201da3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.014607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.021013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.022604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.034109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 969/1933] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:40 2022] PUT /identity/v3/projects/c1574d38a0f847a497811efa97a6681f/users/ac66eb0e72784a8b948b63108f201da3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:41.036842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 964/1933] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:41.038086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:41.095156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:41.123406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08884c3-97df-420f-9740-8ddad876a9b0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:41.128489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6956f-dc03-4c94-96d8-f2db734c3855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:41.129252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6956f-dc03-4c94-96d8-f2db734c3855 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.129560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6956f-dc03-4c94-96d8-f2db734c3855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.129830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6956f-dc03-4c94-96d8-f2db734c3855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.130561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6956f-dc03-4c94-96d8-f2db734c3855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.132168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6956f-dc03-4c94-96d8-f2db734c3855 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.147410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 970/1934] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:41.152969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:41.175585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bf32-f5b5-433c-be4f-d6876d06d4fd service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:41.181053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7489553-806c-4f8d-bbdd-d5cff7449ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:41.182435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7489553-806c-4f8d-bbdd-d5cff7449ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.182705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7489553-806c-4f8d-bbdd-d5cff7449ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.183190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7489553-806c-4f8d-bbdd-d5cff7449ec6 admin admin] PATH_INFO: `/v3/projects/c1574d38a0f847a497811efa97a6681f/users/ac66eb0e72784a8b948b63108f201da3/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.185598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7489553-806c-4f8d-bbdd-d5cff7449ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1574d38a0f847a497811efa97a6681f, user_id=ac66eb0e72784a8b948b63108f201da3, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.188758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7489553-806c-4f8d-bbdd-d5cff7449ec6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.212600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 965/1935] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:41 2022] PUT /identity/v3/projects/c1574d38a0f847a497811efa97a6681f/users/ac66eb0e72784a8b948b63108f201da3/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:41.225035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.225308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.225585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.285214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] MFA Rules not processed for user `ac66eb0e72784a8b948b63108f201da3`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:41.289188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:41.436807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d685-de4f-40b9-9b40-9856b0e91736 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.437110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d685-de4f-40b9-9b40-9856b0e91736 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.437392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d685-de4f-40b9-9b40-9856b0e91736 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.439464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 966/1936] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:33:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:33:41.441855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 971/1937] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:41.464883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.465343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.465449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.501041 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:41.502857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:41.503875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:41.505691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:41.604585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 967/1938] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:41.605366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:41.608803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:41.624555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e0c69-7d6a-4102-959d-8df872e6f279 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:41.632948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:41.633677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.633967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.634246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.647402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.649256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.662574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 972/1939] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:41.691216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:41.693066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:41.745628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17124652-301e-4f84-a85a-b6c6fa0af9c2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:41.756906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e96f4-203d-4d2b-8de3-899a7971543b service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:41.766996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:41.771690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.772101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.773093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] PATH_INFO: `/v3/projects/790a398668664cdb83089761d62a0cc6` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.774514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] RBAC: Authorizing `identity:delete_project(project_id=790a398668664cdb83089761d62a0cc6)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.777972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.782057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:41.783085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:41.783375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:41.783854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:41.809146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:41.815534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:41.842707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 973/1940] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:41.884817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:41.893384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790a398668664cdb83089761d62a0cc6', 'request_id': 'req-484fb012-dc39-4d19-a63e-590d4ad5072c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:41.917434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790a398668664cdb83089761d62a0cc6', 'request_id': 'req-484fb012-dc39-4d19-a63e-590d4ad5072c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:41.924742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790a398668664cdb83089761d62a0cc6', 'request_id': 'req-484fb012-dc39-4d19-a63e-590d4ad5072c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:41.944308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] The token cache is being invalidate because project 790a398668664cdb83089761d62a0cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:41.944705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:41.947990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:41.949559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 968/1941] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:41 2022] DELETE /identity/v3/projects/790a398668664cdb83089761d62a0cc6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:41.954037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:41.968620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bca914-f349-4c17-9863-111fbfc794dc service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:42.069884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:42.098443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:42.099562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:42.100106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:42.100557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] PATH_INFO: `/v3/users/22e3df884b8847dcb70b57f82376f5ec` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:42.104471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e3df884b8847dcb70b57f82376f5ec)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:42.107887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:42.200126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e3df884b8847dcb70b57f82376f5ec'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.201411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e3df884b8847dcb70b57f82376f5ec'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.226055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e3df884b8847dcb70b57f82376f5ec'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.227454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e3df884b8847dcb70b57f82376f5ec'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.248671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e3df884b8847dcb70b57f82376f5ec'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.268821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 974/1942] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:41 2022] DELETE /identity/v3/users/22e3df884b8847dcb70b57f82376f5ec => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:42.757321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:42.759420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:42.777600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fb012-dc39-4d19-a63e-590d4ad5072c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:42.884674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:42.885829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:42.886400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:42.886603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] PATH_INFO: `/v3/projects/31dcc19c10e8454f874717f7bd974417` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:42.888209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31dcc19c10e8454f874717f7bd974417)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:42.890220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:42.943873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:42.945488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:42.957611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31dcc19c10e8454f874717f7bd974417', 'request_id': 'req-9c911842-bd7f-45ed-8863-4ae83fe2c316'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.960002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920df9-9fc1-4663-8b1d-2935fe617c30 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:42.968360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31dcc19c10e8454f874717f7bd974417', 'request_id': 'req-9c911842-bd7f-45ed-8863-4ae83fe2c316'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.974698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31dcc19c10e8454f874717f7bd974417', 'request_id': 'req-9c911842-bd7f-45ed-8863-4ae83fe2c316'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.986932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] The token cache is being invalidate because project 31dcc19c10e8454f874717f7bd974417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:42.987300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.988051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c911842-bd7f-45ed-8863-4ae83fe2c316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:42.990118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 969/1943] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:42 2022] DELETE /identity/v3/projects/31dcc19c10e8454f874717f7bd974417 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:42.994647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcc030-b05f-40c5-8c12-a3104ae1ef2e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:42.994934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcc030-b05f-40c5-8c12-a3104ae1ef2e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:42.995276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcc030-b05f-40c5-8c12-a3104ae1ef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.014686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-88bcc030-b05f-40c5-8c12-a3104ae1ef2e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:43.015921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-88bcc030-b05f-40c5-8c12-a3104ae1ef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:43.037191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.038419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.039865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.040374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] PATH_INFO: `/v3/users/a64f75f4402740308148fd7e33c0ccf8` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.042349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64f75f4402740308148fd7e33c0ccf8)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.045403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.088709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 970/1944] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:43.096563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.096883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.097194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.103613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:43.114505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64f75f4402740308148fd7e33c0ccf8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:43.115249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64f75f4402740308148fd7e33c0ccf8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:43.121757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:43.124157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:43.131712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64f75f4402740308148fd7e33c0ccf8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:43.132446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64f75f4402740308148fd7e33c0ccf8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:43.134212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 971/1945] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:43.144562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.149409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-139a5ca8-145e-48bc-b6ed-31af490eb78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64f75f4402740308148fd7e33c0ccf8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:43.155936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:43.162776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700609e8-0725-455f-a1ec-010a2ee07294 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.171357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 975/1946] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:42 2022] DELETE /identity/v3/users/a64f75f4402740308148fd7e33c0ccf8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:43.177837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.179388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.179597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.184008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.184789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.185262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.185553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.193829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:43.194384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.196786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.216120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 972/1947] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:43.221628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.225626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:43.227082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:43.239938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c850807-66c6-4023-a235-fe584a49aded admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.318631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 976/1948] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:43.325969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.328460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:43.333600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.334625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.334975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.335318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.336659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.339033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.397531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 973/1949] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:43.398796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.420822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04651ae8-35a2-49db-ae3f-f60b551491f7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.501310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725474-be17-4a60-9083-595314094d68 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.513291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.516611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.516837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.517085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.517849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.519767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.521677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.522977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.523161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.523508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.537958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.539715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.557370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 974/1950] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:43.561646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 977/1951] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:43.561787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.572164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad17e8-29c0-4fcc-8001-954e5334532d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.577104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.577617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.577906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.578177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.578917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.580523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.591151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 975/1952] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:43.595346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.605173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a69e-0623-40b2-b01c-e07ab2807857 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.610007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea141e2-f642-42c3-b8fd-b701a305fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.610719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea141e2-f642-42c3-b8fd-b701a305fe38 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.611001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea141e2-f642-42c3-b8fd-b701a305fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.611300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea141e2-f642-42c3-b8fd-b701a305fe38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.612013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea141e2-f642-42c3-b8fd-b701a305fe38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.613662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea141e2-f642-42c3-b8fd-b701a305fe38 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.624601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 978/1953] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:43.629092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.639399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdcfa6-abd8-4ba3-b0b0-5e99582f81a1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.644270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.645196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.645476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.645745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] PATH_INFO: `/v3/projects/128a26415f3c4770a3318e42db48264e/users/e8627e50f59746a6b9273a59f854db3b/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.661829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=128a26415f3c4770a3318e42db48264e, user_id=e8627e50f59746a6b9273a59f854db3b, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.664602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.679031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 976/1954] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:43 2022] PUT /identity/v3/projects/128a26415f3c4770a3318e42db48264e/users/e8627e50f59746a6b9273a59f854db3b/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:43.691181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.691606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.691889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.726974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] MFA Rules not processed for user `e8627e50f59746a6b9273a59f854db3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:43.728243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:43.759239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:43.797686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 979/1955] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:43.802429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.829020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:43.845659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.847156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:43.884604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5736b84-c8f3-42f4-8a1a-8cb1b94d91b6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.892294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.892294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.893132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.893422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.894167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.896531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:43.930162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed8b1-1aac-40c9-a633-eccf7ca38367 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:43.930669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 977/1956] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:43.935231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:43.990152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:43.991036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:43.991320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:43.991602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] PATH_INFO: `/v3/projects/367c3bb8852f4ddeb87eaa809c8c1fd5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:43.994908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367c3bb8852f4ddeb87eaa809c8c1fd5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:43.996878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.004005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97899705-aedb-4a18-be61-606a6291d474 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.010768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.011651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.011762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.012259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.013167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.015672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.047766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 978/1957] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:43 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:44.048865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.053308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367c3bb8852f4ddeb87eaa809c8c1fd5', 'request_id': 'req-15259df5-b188-4ce0-bcbb-d35d182dc945'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.058635 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace7e2e-0d70-46eb-b0d8-c2e1511b2c2a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.060212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367c3bb8852f4ddeb87eaa809c8c1fd5', 'request_id': 'req-15259df5-b188-4ce0-bcbb-d35d182dc945'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.066047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367c3bb8852f4ddeb87eaa809c8c1fd5', 'request_id': 'req-15259df5-b188-4ce0-bcbb-d35d182dc945'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.079931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] The token cache is being invalidate because project 367c3bb8852f4ddeb87eaa809c8c1fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:44.080228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.080941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.082536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 980/1958] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:43 2022] DELETE /identity/v3/projects/367c3bb8852f4ddeb87eaa809c8c1fd5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:44.086600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.087977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.101164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:44.122498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.123190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.123477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.123756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.124493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.126151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.136860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 979/1959] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:44.143206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.145052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.157495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d024-8c53-4ab5-a7e2-15491e775f99 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.162671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.163149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15259df5-b188-4ce0-bcbb-d35d182dc945 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.163563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.163841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.164159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] PATH_INFO: `/v3/projects/7d232d71145c449d81fc3dd40dd08bb3/users/52bbc0466c504997803fa95b2641429e/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.174957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.176021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.176297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.176565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] PATH_INFO: `/v3/users/ac66eb0e72784a8b948b63108f201da3` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.177976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac66eb0e72784a8b948b63108f201da3)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.179940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.181415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d232d71145c449d81fc3dd40dd08bb3, user_id=52bbc0466c504997803fa95b2641429e, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.184231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.198194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 980/1960] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:44 2022] PUT /identity/v3/projects/7d232d71145c449d81fc3dd40dd08bb3/users/52bbc0466c504997803fa95b2641429e/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:44.202571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.213379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:44.234235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac66eb0e72784a8b948b63108f201da3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.234924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac66eb0e72784a8b948b63108f201da3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.255062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac66eb0e72784a8b948b63108f201da3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.255789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac66eb0e72784a8b948b63108f201da3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.265416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac66eb0e72784a8b948b63108f201da3'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.279712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 981/1961] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:44 2022] DELETE /identity/v3/users/ac66eb0e72784a8b948b63108f201da3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:44.281589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63923b-a26f-4d7c-bc5e-ede65a95c595 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.361898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac55c9-844a-409d-9504-67c134fc5e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.362600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac55c9-844a-409d-9504-67c134fc5e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.362885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac55c9-844a-409d-9504-67c134fc5e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.363160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac55c9-844a-409d-9504-67c134fc5e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.363901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac55c9-844a-409d-9504-67c134fc5e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.365569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac55c9-844a-409d-9504-67c134fc5e1b admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.377351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 981/1962] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:44.381545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.383059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.397673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297df57-89fe-4d73-811d-8a7709a2af4d admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.403192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.404281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.404575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.404937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] PATH_INFO: `/v3/projects/7d232d71145c449d81fc3dd40dd08bb3/users/52bbc0466c504997803fa95b2641429e/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.406964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d232d71145c449d81fc3dd40dd08bb3, user_id=52bbc0466c504997803fa95b2641429e, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.409684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.421083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 982/1963] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:44 2022] PUT /identity/v3/projects/7d232d71145c449d81fc3dd40dd08bb3/users/52bbc0466c504997803fa95b2641429e/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:44.432793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.433080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.433356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.466892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] MFA Rules not processed for user `52bbc0466c504997803fa95b2641429e`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:44.468188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.578040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 982/1964] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:44.605584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.613270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.622478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.624086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.725101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:44.745451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4665ec-1c3e-4c1d-b98c-86f1a37d3871 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.750570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.751525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.752004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.752292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] PATH_INFO: `/v3/projects/c1574d38a0f847a497811efa97a6681f` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.753891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1574d38a0f847a497811efa97a6681f)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.755886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.756393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fce71-7aaa-437b-a29e-3e9db8d9e508 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.766445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f923839-85e5-4634-809f-e91505120ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.767256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f923839-85e5-4634-809f-e91505120ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.767550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f923839-85e5-4634-809f-e91505120ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.767914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f923839-85e5-4634-809f-e91505120ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.774772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:44.789308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f923839-85e5-4634-809f-e91505120ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:44.791207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f923839-85e5-4634-809f-e91505120ca3 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:44.804679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 983/1965] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:44.826812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1574d38a0f847a497811efa97a6681f', 'request_id': 'req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.834179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1574d38a0f847a497811efa97a6681f', 'request_id': 'req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.842824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1574d38a0f847a497811efa97a6681f', 'request_id': 'req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.861963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] The token cache is being invalidate because project c1574d38a0f847a497811efa97a6681f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:44.862529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.863237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:44.868216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 983/1966] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:44 2022] DELETE /identity/v3/projects/c1574d38a0f847a497811efa97a6681f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:44.870227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2b7df-01d1-4250-aacd-9cd22a83c48c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.870553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2b7df-01d1-4250-aacd-9cd22a83c48c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.870857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2b7df-01d1-4250-aacd-9cd22a83c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.896528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-56e2b7df-01d1-4250-aacd-9cd22a83c48c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:44.897806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-56e2b7df-01d1-4250-aacd-9cd22a83c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.923582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:44.924916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.964210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 984/1967] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:44.972323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.972496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.972758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.985109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70eaac-0659-4038-ba32-a4dbf5daf1a4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:44.990330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:44.991029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:44.991634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:44.991634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:44.991830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:44.993190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:44.993462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:45.001770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 985/1968] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:45.006556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.016470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6759c0-4ed5-49f1-9ef1-c49784e2edd2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.021999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.022711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.022961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.023238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.023973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:45.024305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.025976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.042081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 986/1969] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:33:45.046605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.057919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b527bd-7dcb-4394-b799-f726cac89e9e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.070672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.071724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.072405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.072714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.072818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.072997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.073686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.076025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.084585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 984/1970] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:45.116481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 987/1971] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:45 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:45.117835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.137188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40590e-ccb2-48a0-a3ec-3526149d3cfa service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.218542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:45.230757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.236693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.239327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.239327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.239327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.240908 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.280574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 985/1972] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:45 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:45.284645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.295425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a45125-46dc-47a6-b0cb-474eab0b0072 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.301167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.301918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.302270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.302547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.303384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.305401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.320662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 988/1973] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:45.323266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.333966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f756c-bef3-4240-895d-f212679d86b0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.338631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.339313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.339592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.339861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.340716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.342319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.352519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 986/1974] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:45.357170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.367571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051995b2-f253-4613-b978-c4dd3de4ef62 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.372663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110be25-115b-47ff-8366-3823c0312ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.373493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110be25-115b-47ff-8366-3823c0312ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.373775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110be25-115b-47ff-8366-3823c0312ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.374045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110be25-115b-47ff-8366-3823c0312ab7 admin admin] PATH_INFO: `/v3/projects/3e86eb149c75441cbd04d4d4d03302ea/users/398669ccb0ef4b188c5178831795101e/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.390572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110be25-115b-47ff-8366-3823c0312ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e86eb149c75441cbd04d4d4d03302ea, user_id=398669ccb0ef4b188c5178831795101e, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.393460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110be25-115b-47ff-8366-3823c0312ab7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.400586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.402717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:45.415209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 989/1975] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:45 2022] PUT /identity/v3/projects/3e86eb149c75441cbd04d4d4d03302ea/users/398669ccb0ef4b188c5178831795101e/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:45.425241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f55f27-a078-4c0e-87c2-4eb1360894dd admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.425770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.426009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.426280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.459536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] MFA Rules not processed for user `398669ccb0ef4b188c5178831795101e`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:45.460919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:45.492244 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.493307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.493621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.493898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] PATH_INFO: `/v3/users/e8627e50f59746a6b9273a59f854db3b` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.495404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8627e50f59746a6b9273a59f854db3b)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.498198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.524583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:45.538691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 990/1976] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:45.551656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.556503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8627e50f59746a6b9273a59f854db3b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:45.557223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8627e50f59746a6b9273a59f854db3b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:45.570794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8627e50f59746a6b9273a59f854db3b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:45.571495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8627e50f59746a6b9273a59f854db3b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:45.579595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-40682fb4-83ca-4b4a-9d7b-eb89ce10d599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8627e50f59746a6b9273a59f854db3b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:45.598508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 987/1977] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:45 2022] DELETE /identity/v3/users/e8627e50f59746a6b9273a59f854db3b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:45.602484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.602845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.603127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.613333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f209f7-17a3-447c-bf37-071576833f81 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.621496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:45.623466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:45.672601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.673416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.673758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.674091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.675980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:45.677334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:45.697531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 988/1978] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:45.707779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:45.710035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:45.747734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.749479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.760489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 991/1979] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:45.781165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337de368-b9dd-40ca-a0db-7e071e19e9e5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:45.785295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:45.786149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:45.786524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:45.786981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:45.798926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:45.800380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:45.811982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 989/1980] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:46.244142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:46.252962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:46.275481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62c00d-ae1c-4ee7-8a22-123493e013bf service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:46.293298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:46.294968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:46.296012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:46.296433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] PATH_INFO: `/v3/projects/128a26415f3c4770a3318e42db48264e` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:46.300373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=128a26415f3c4770a3318e42db48264e)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:46.304980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:46.374017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128a26415f3c4770a3318e42db48264e', 'request_id': 'req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.382479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128a26415f3c4770a3318e42db48264e', 'request_id': 'req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.384760 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:46.390373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128a26415f3c4770a3318e42db48264e', 'request_id': 'req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.402458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] The token cache is being invalidate because project 128a26415f3c4770a3318e42db48264e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:46.402747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.403777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.405392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 992/1981] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:46 2022] DELETE /identity/v3/projects/128a26415f3c4770a3318e42db48264e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:46.409475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:46.419622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295825bf-ea1f-4451-af0e-eb43752b1066 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:46.475579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:46.476499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:46.476814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:46.477090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] PATH_INFO: `/v3/users/52bbc0466c504997803fa95b2641429e` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:46.478284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bbc0466c504997803fa95b2641429e)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:46.480212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:46.537088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bbc0466c504997803fa95b2641429e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.537856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bbc0466c504997803fa95b2641429e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.551174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bbc0466c504997803fa95b2641429e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.551819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bbc0466c504997803fa95b2641429e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.560438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1fb0f58e-5939-4892-8bab-80f9f73e92ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bbc0466c504997803fa95b2641429e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:46.573594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 990/1982] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:46 2022] DELETE /identity/v3/users/52bbc0466c504997803fa95b2641429e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:46.877362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:46.879590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:46.895395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fbd80-e89d-4fc1-9003-b39b81e6b845 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:46.952303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:46.953152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:46.953429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:46.953695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] PATH_INFO: `/v3/projects/7d232d71145c449d81fc3dd40dd08bb3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:46.954982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d232d71145c449d81fc3dd40dd08bb3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:46.956919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.006357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d232d71145c449d81fc3dd40dd08bb3', 'request_id': 'req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.014253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d232d71145c449d81fc3dd40dd08bb3', 'request_id': 'req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.020188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d232d71145c449d81fc3dd40dd08bb3', 'request_id': 'req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.031606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] The token cache is being invalidate because project 7d232d71145c449d81fc3dd40dd08bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:47.031928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.032590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ef4b4ab0-e05b-4e3c-b782-1ffad327e8f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.034214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 993/1983] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:46 2022] DELETE /identity/v3/projects/7d232d71145c449d81fc3dd40dd08bb3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:47.039311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.039607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.039889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.059693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:47.060978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:47.074132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:47.127462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 991/1984] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:47.136168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.136522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.136969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.165405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:47.166628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:47.178675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 994/1985] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:47.183289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.194034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dde14-7478-4772-9f4d-fafc69911c5a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.198711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.199375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.199649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.199930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.200991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:47.201333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.202989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.217187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 992/1986] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:47.221413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.231456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485c4e7-c884-4d1b-b6f8-a92a2fcf5793 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.236614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.237318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.237816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.238103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.238812 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.240782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.284831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 995/1987] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:47 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:47.286858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.298318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112749-fd1d-44db-838b-7e7217b2fb5e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.366907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.367592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.367875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.368151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.368875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.371847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.397291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 993/1988] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:47 2022] POST /identity/v3/users => generated 280 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:47.402348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.419848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7758f6-9b72-4067-a911-b288d89257af admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.429147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.429905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.430236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.430663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.431757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.434229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.445354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 996/1989] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:47.449402 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.459820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873f6c-4f09-43af-89ee-43f07f649a3c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.465014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3095e-6fe8-4408-b41d-e59e2a5f46a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.465803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3095e-6fe8-4408-b41d-e59e2a5f46a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.466144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3095e-6fe8-4408-b41d-e59e2a5f46a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.466456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3095e-6fe8-4408-b41d-e59e2a5f46a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.467163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3095e-6fe8-4408-b41d-e59e2a5f46a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.468798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3095e-6fe8-4408-b41d-e59e2a5f46a3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.479405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 994/1990] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:47.484150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.494424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fe8c1-25af-4fa3-9501-2468cab9c92f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.499263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.500152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.500432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.500726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] PATH_INFO: `/v3/projects/2af9b847ccec414fa9fdc79dec02e96a/users/3957ab084bdf4b3bb1684a7676230492/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.516010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af9b847ccec414fa9fdc79dec02e96a, user_id=3957ab084bdf4b3bb1684a7676230492, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.518712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.532845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 997/1991] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:47 2022] PUT /identity/v3/projects/2af9b847ccec414fa9fdc79dec02e96a/users/3957ab084bdf4b3bb1684a7676230492/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:47.543197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db82614-5157-4221-8446-454028c4d282 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.543487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db82614-5157-4221-8446-454028c4d282 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.543902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db82614-5157-4221-8446-454028c4d282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.572914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2db82614-5157-4221-8446-454028c4d282 None None] MFA Rules not processed for user `3957ab084bdf4b3bb1684a7676230492`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:47.574156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2db82614-5157-4221-8446-454028c4d282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:47.640836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 995/1992] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:47.682007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.683947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:47.699352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:47.744148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257f4ba-5c19-42c1-9d48-edf819173338 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.748655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a145-ec0e-4481-a947-0d9da8e460cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.749329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a145-ec0e-4481-a947-0d9da8e460cb service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.749612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a145-ec0e-4481-a947-0d9da8e460cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.749884 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a145-ec0e-4481-a947-0d9da8e460cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.758932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26a145-ec0e-4481-a947-0d9da8e460cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.760373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26a145-ec0e-4481-a947-0d9da8e460cb service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.769614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 998/1993] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:47 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:47.783897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db82614-5157-4221-8446-454028c4d282 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:47.799700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:47.851513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db82614-5157-4221-8446-454028c4d282 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:47.856419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:47.857345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.857611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.857882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] PATH_INFO: `/v3/users/3957ab084bdf4b3bb1684a7676230492` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.859346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3957ab084bdf4b3bb1684a7676230492)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:47.861949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:47.913533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3957ab084bdf4b3bb1684a7676230492'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.914366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3957ab084bdf4b3bb1684a7676230492'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.931005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3957ab084bdf4b3bb1684a7676230492'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.931745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3957ab084bdf4b3bb1684a7676230492'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.939728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3957ab084bdf4b3bb1684a7676230492'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:47.952269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 996/1994] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:47 2022] DELETE /identity/v3/users/3957ab084bdf4b3bb1684a7676230492 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:47.956762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:47.957053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:47.957331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:47.975428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:47.976649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:48.036387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 999/1995] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:48.047919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:48.050628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:48.117479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4ee6-eb22-47eb-8ff3-f0811095e703 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:48.121556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81430d35-83e6-4129-a314-189b009e027c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:48.122241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81430d35-83e6-4129-a314-189b009e027c service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.122523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81430d35-83e6-4129-a314-189b009e027c service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.122795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81430d35-83e6-4129-a314-189b009e027c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.133014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81430d35-83e6-4129-a314-189b009e027c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:48.134469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81430d35-83e6-4129-a314-189b009e027c service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:48.144270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 997/1996] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:48.470871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:48.472445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:48.501243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac26ae-e4aa-444f-97d8-4d6e532fff02 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:48.511152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:48.511877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.512154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.512443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] PATH_INFO: `/v3/projects/2af9b847ccec414fa9fdc79dec02e96a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.521185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af9b847ccec414fa9fdc79dec02e96a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:48.523283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:48.583539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af9b847ccec414fa9fdc79dec02e96a', 'request_id': 'req-d996b908-d49e-4fa4-99b7-57b8f7a1af88'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:48.591603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af9b847ccec414fa9fdc79dec02e96a', 'request_id': 'req-d996b908-d49e-4fa4-99b7-57b8f7a1af88'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:48.598510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af9b847ccec414fa9fdc79dec02e96a', 'request_id': 'req-d996b908-d49e-4fa4-99b7-57b8f7a1af88'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:48.619132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] The token cache is being invalidate because project 2af9b847ccec414fa9fdc79dec02e96a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:48.619702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:48.620659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d996b908-d49e-4fa4-99b7-57b8f7a1af88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:48.623517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1000/1997] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:48 2022] DELETE /identity/v3/projects/2af9b847ccec414fa9fdc79dec02e96a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:48.628578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc02d21-9292-4208-b748-198519872b15 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.629346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc02d21-9292-4208-b748-198519872b15 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.629346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc02d21-9292-4208-b748-198519872b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.649062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-dcc02d21-9292-4208-b748-198519872b15 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:48.650275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-dcc02d21-9292-4208-b748-198519872b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:48.686232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:48.733707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 998/1998] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:48.742113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.742484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.742805 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.764962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:48.766349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:48.775569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1001/1999] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:48.780564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc02d21-9292-4208-b748-198519872b15 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:48.790545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc02d21-9292-4208-b748-198519872b15 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:48.795503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:48.806732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.806732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.806732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.806732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:48.806732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:48.806732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:48.814077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 999/2000] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:48.818710 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:48.828690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411ceb1-37cb-48c5-97ea-1e18733a5c61 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:48.834768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:48.835432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.835713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.835986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.836707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:48.838636 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:48.868232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1002/2001] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:48 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:48.873166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:48.884228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0bbb6-e5da-4305-80f5-628a753e0005 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:48.962097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:48.962873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:48.963212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:48.963516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:48.964416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:48.966833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:48.990358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1000/2002] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:48 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:48.994554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:49.005866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28fb6-c0ce-4377-9480-064855868aa2 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:49.010587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:49.012196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:49.012531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:49.013184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:49.013937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:49.015556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:49.021970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:49.032565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1003/2003] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:49.034073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:49.045842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613acf-4ea2-4b12-a60e-68dc8c5e0db9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:49.051004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f9ce1-cd32-46fb-af89-5104b5945c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:49.051766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f9ce1-cd32-46fb-af89-5104b5945c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:49.052150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f9ce1-cd32-46fb-af89-5104b5945c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:49.052493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f9ce1-cd32-46fb-af89-5104b5945c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:49.053336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f9ce1-cd32-46fb-af89-5104b5945c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:49.055077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f9ce1-cd32-46fb-af89-5104b5945c66 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:49.067031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1001/2004] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:49.071170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:49.081584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88179908-a294-41c4-947c-50482282fc09 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:49.086258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:49.087144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:49.087420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:49.087690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] PATH_INFO: `/v3/projects/17ecb6953f7f4a689b4bb605832c5c6c/users/c3f271f137054858902257e22192907d/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:49.104931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ecb6953f7f4a689b4bb605832c5c6c, user_id=c3f271f137054858902257e22192907d, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:49.107907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:49.124565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1004/2005] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:49 2022] PUT /identity/v3/projects/17ecb6953f7f4a689b4bb605832c5c6c/users/c3f271f137054858902257e22192907d/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:49.135060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:49.135345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:49.135616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:49.165817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] MFA Rules not processed for user `c3f271f137054858902257e22192907d`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:49.167065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:49.240354 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1002/2006] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:49.251103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:49.254170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:49.318127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acf79-d893-42e0-80ab-c685d43bdcd3 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:49.321927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901f8a-5f5f-45b4-a435-fafd14a8f5a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:49.322670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901f8a-5f5f-45b4-a435-fafd14a8f5a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:49.322958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901f8a-5f5f-45b4-a435-fafd14a8f5a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:49.323336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901f8a-5f5f-45b4-a435-fafd14a8f5a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:49.335427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97901f8a-5f5f-45b4-a435-fafd14a8f5a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:49.337025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97901f8a-5f5f-45b4-a435-fafd14a8f5a6 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:49.346826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1005/2007] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:51.587009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:51.588580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:51.622198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:51.654559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325cb71-5125-4cd9-9fe2-08d9e2d12771 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:51.660211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:51.661139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:51.661436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:51.661712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] PATH_INFO: `/v3/users/398669ccb0ef4b188c5178831795101e` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:51.663027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=398669ccb0ef4b188c5178831795101e)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:51.665043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:51.717296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398669ccb0ef4b188c5178831795101e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:51.717966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398669ccb0ef4b188c5178831795101e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:51.729593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398669ccb0ef4b188c5178831795101e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:51.730241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398669ccb0ef4b188c5178831795101e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:51.737689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398669ccb0ef4b188c5178831795101e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:51.749850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1003/2008] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:51 2022] DELETE /identity/v3/users/398669ccb0ef4b188c5178831795101e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:51.754622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:51.755143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:51.755143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:51.772873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:51.774198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:51.839149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1006/2009] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:51.849215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:51.850990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:51.912498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af134e-ff6d-4686-b285-d33053ec1a02 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:51.916837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7b1b5-f1de-4050-ae9a-43512e7b9fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:51.917511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7b1b5-f1de-4050-ae9a-43512e7b9fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:51.917793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7b1b5-f1de-4050-ae9a-43512e7b9fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:51.918773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7b1b5-f1de-4050-ae9a-43512e7b9fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:51.929399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7b1b5-f1de-4050-ae9a-43512e7b9fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:51.930903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7b1b5-f1de-4050-ae9a-43512e7b9fb0 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:51.941944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1004/2010] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:52.283565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:52.285060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:52.304182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbee83-23df-426d-9695-fd08c8b510b3 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:52.327431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:52.329955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:52.330442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:52.330877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] PATH_INFO: `/v3/projects/3e86eb149c75441cbd04d4d4d03302ea` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:52.333715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e86eb149c75441cbd04d4d4d03302ea)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:52.338703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:52.454338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e86eb149c75441cbd04d4d4d03302ea', 'request_id': 'req-24c858e8-3560-4238-b01d-eab3c19464c9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.474601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e86eb149c75441cbd04d4d4d03302ea', 'request_id': 'req-24c858e8-3560-4238-b01d-eab3c19464c9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.485583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e86eb149c75441cbd04d4d4d03302ea', 'request_id': 'req-24c858e8-3560-4238-b01d-eab3c19464c9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.527560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] The token cache is being invalidate because project 3e86eb149c75441cbd04d4d4d03302ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:52.528371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.529097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.535517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1007/2011] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:52 2022] DELETE /identity/v3/projects/3e86eb149c75441cbd04d4d4d03302ea => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:52.541386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4770-2d07-45f5-beec-f192ceffd12b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:52.542261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4770-2d07-45f5-beec-f192ceffd12b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:52.543850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4770-2d07-45f5-beec-f192ceffd12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:52.574064 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ac7d4770-2d07-45f5-beec-f192ceffd12b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:52.584696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ac7d4770-2d07-45f5-beec-f192ceffd12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:52.672251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:52.674115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:52.685168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:52.765613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:52.797495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1005/2012] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:52.806395 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:52.806880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:52.807931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:52.826663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c858e8-3560-4238-b01d-eab3c19464c9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:52.832614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:52.833860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:52.834352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:52.835054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] PATH_INFO: `/v3/users/15513c3389ea4c2097c03e118b1d0d2b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:52.837076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15513c3389ea4c2097c03e118b1d0d2b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:52.839658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:52.852831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:52.856652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:52.877045 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1006/2013] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:52.882796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:52.899982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe2ac-2486-4a04-8688-97a8a6c8de59 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:52.905669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:52.906618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:52.906940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:52.907691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:52.908136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:52.908449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:52.910156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:52.914804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15513c3389ea4c2097c03e118b1d0d2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.916218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15513c3389ea4c2097c03e118b1d0d2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.930937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1007/2014] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:52.933230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15513c3389ea4c2097c03e118b1d0d2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.935027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:52.937259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15513c3389ea4c2097c03e118b1d0d2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.938088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:52.946533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-dbc1946a-9444-4d62-90ab-b2342e7c28a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15513c3389ea4c2097c03e118b1d0d2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:52.961769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e95c19-3d0a-43d7-ad3b-6538b0340701 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:52.964701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1008/2015] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:52 2022] DELETE /identity/v3/users/15513c3389ea4c2097c03e118b1d0d2b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:52.966851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:52.967857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:52.968219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:52.991097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:52.993459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.028525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.029394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.029698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.030107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.030921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.033144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.067828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1009/2016] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:53.068052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1008/2017] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:52 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:53.073411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.087100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.093925 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b1bc6-a9ef-4a83-9aaf-c8b6838beab5 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.094116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.119886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:53.231954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.232791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.233069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.233511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.235601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.244376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b610c32-3888-424c-a185-9b9fefc38d20 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.248018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.252950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.253637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.253918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.254238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.273382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.275350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.291303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1009/2018] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:53.301788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1010/2019] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:53 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:53.302603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.314117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5e8d8-d267-47db-976d-66876a029479 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.319239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.320505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.321327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.321327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.321645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.323371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.336852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1010/2020] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:53.340133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.354214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec08d0-0506-41b8-8f47-42fa399a9c84 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.361501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.362641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.363346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.363626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.363897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.364861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.366558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.368660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.385473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1011/2021] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:53.386630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.392860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dbb50-5369-4ca7-b46a-b471d5178e63 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.403930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.404387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922622f-3d18-4942-9b6f-5cb97b90735c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.408650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.409738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.410475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] PATH_INFO: `/v3/users/a8e8772662c14a2d9cfa8a5f39bd0ece` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.411877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed218a-b6ee-4071-8712-1fd1a195970b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.412969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e8772662c14a2d9cfa8a5f39bd0ece)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.413713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed218a-b6ee-4071-8712-1fd1a195970b admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.413713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed218a-b6ee-4071-8712-1fd1a195970b admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.413713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed218a-b6ee-4071-8712-1fd1a195970b admin admin] PATH_INFO: `/v3/projects/48748543262c417d9089326f428e6b63/users/c969fad4feb7468abd62714899d3fc2b/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.416278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.434951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed218a-b6ee-4071-8712-1fd1a195970b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48748543262c417d9089326f428e6b63, user_id=c969fad4feb7468abd62714899d3fc2b, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:53.438573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed218a-b6ee-4071-8712-1fd1a195970b admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:53.462087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1012/2022] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:53 2022] PUT /identity/v3/projects/48748543262c417d9089326f428e6b63/users/c969fad4feb7468abd62714899d3fc2b/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:53.473693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.474025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.474302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.511919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] MFA Rules not processed for user `c969fad4feb7468abd62714899d3fc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:53.513366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.513858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:53.516867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e8772662c14a2d9cfa8a5f39bd0ece'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:53.517915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e8772662c14a2d9cfa8a5f39bd0ece'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:53.538042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e8772662c14a2d9cfa8a5f39bd0ece'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:53.539263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e8772662c14a2d9cfa8a5f39bd0ece'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:53.550882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-56bd05ae-0ec9-4e57-8a1a-1a2208f23138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e8772662c14a2d9cfa8a5f39bd0ece'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:53.566777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1011/2023] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:53 2022] DELETE /identity/v3/users/a8e8772662c14a2d9cfa8a5f39bd0ece => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:53.571383 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.571678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.571997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.603064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1013/2024] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:53.616425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.618668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.621294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:53.622992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.766888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1012/2025] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:33:53.786673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:53.789930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:53.815878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315438b5-2659-4628-9088-3ea1f274e482 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.845371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:53.919249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.920475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.921096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.921671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:53.973618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08948d-a9af-46bd-9d00-e0852ae74f86 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:53.979829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:53.983150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:53.983693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:53.984553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:54.013072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:54.020589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:54.048018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:54.049189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1013/2026] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:54.050563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:54.065335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:54.066791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:54.067345 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1014/2027] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:33:54.085079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7803c1-c38c-4c5e-b94c-d552a38fa8c3 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:54.096687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:54.097874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:54.098153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:54.098424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] PATH_INFO: `/v3/projects/555156e0df76492ea8eee7d8eca75a12` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:54.112823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=555156e0df76492ea8eee7d8eca75a12)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:54.114789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:54.222848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555156e0df76492ea8eee7d8eca75a12', 'request_id': 'req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.230491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555156e0df76492ea8eee7d8eca75a12', 'request_id': 'req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.242845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:54.245139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555156e0df76492ea8eee7d8eca75a12', 'request_id': 'req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.266950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] The token cache is being invalidate because project 555156e0df76492ea8eee7d8eca75a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:54.267576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.268543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.273301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1014/2028] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:54 2022] DELETE /identity/v3/projects/555156e0df76492ea8eee7d8eca75a12 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:54.274468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:54.290549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51f7a-5cc0-417a-9c36-0bae1d7be617 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:54.439501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:54.439501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:54.439948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:54.440302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] PATH_INFO: `/v3/users/0a1e1d2ddd8e4463b4254e7c8bb96950` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:54.441887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1e1d2ddd8e4463b4254e7c8bb96950)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:54.444154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:54.588391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:54.591352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:54.643197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1e1d2ddd8e4463b4254e7c8bb96950'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.645284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1e1d2ddd8e4463b4254e7c8bb96950'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.648076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcbd13-de1a-415a-bd0a-cef09444d3d8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:54.658277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:54.659176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:54.659447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:54.660951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] PATH_INFO: `/v3/projects/55a304c67f47459bba1c5de24ea44f1c` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:54.664706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a304c67f47459bba1c5de24ea44f1c)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:54.666582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:54.682440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1e1d2ddd8e4463b4254e7c8bb96950'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.683671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1e1d2ddd8e4463b4254e7c8bb96950'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.707283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1e1d2ddd8e4463b4254e7c8bb96950'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.719707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:54.736830 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1015/2029] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:54 2022] DELETE /identity/v3/users/0a1e1d2ddd8e4463b4254e7c8bb96950 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:54.769819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a304c67f47459bba1c5de24ea44f1c', 'request_id': 'req-c16d84b5-dd37-4cc0-b223-6feac28e312e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.780219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a304c67f47459bba1c5de24ea44f1c', 'request_id': 'req-c16d84b5-dd37-4cc0-b223-6feac28e312e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.789207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a304c67f47459bba1c5de24ea44f1c', 'request_id': 'req-c16d84b5-dd37-4cc0-b223-6feac28e312e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.821317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] The token cache is being invalidate because project 55a304c67f47459bba1c5de24ea44f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:54.822048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.824343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:54.827708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1015/2030] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:54 2022] DELETE /identity/v3/projects/55a304c67f47459bba1c5de24ea44f1c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:54.832714 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:54.834634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:54.935216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0f4c1-a36e-4064-b1f7-4b03f4fd6d19 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:55.006152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:55.007219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.007507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.007782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] PATH_INFO: `/v3/users/ed6695c134ba42a2bb8e5d1f0d8c98a8` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:55.009042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6695c134ba42a2bb8e5d1f0d8c98a8)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:55.011326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:55.093154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6695c134ba42a2bb8e5d1f0d8c98a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.094201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6695c134ba42a2bb8e5d1f0d8c98a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.113284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6695c134ba42a2bb8e5d1f0d8c98a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.114616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6695c134ba42a2bb8e5d1f0d8c98a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.130399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ca0c7d27-3297-45f8-bb67-7e8240f3ee2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6695c134ba42a2bb8e5d1f0d8c98a8'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.154453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1016/2031] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:33:54 2022] DELETE /identity/v3/users/ed6695c134ba42a2bb8e5d1f0d8c98a8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:55.263748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:55.265538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:55.286209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d84b5-dd37-4cc0-b223-6feac28e312e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:55.395719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:55.396777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.397157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.397598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] PATH_INFO: `/v3/projects/243a2af6952540ac8cc6c31bf7cdfb8e` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:55.398928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=243a2af6952540ac8cc6c31bf7cdfb8e)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:55.401135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:55.484411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243a2af6952540ac8cc6c31bf7cdfb8e', 'request_id': 'req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.494166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243a2af6952540ac8cc6c31bf7cdfb8e', 'request_id': 'req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.504450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243a2af6952540ac8cc6c31bf7cdfb8e', 'request_id': 'req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.527592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] The token cache is being invalidate because project 243a2af6952540ac8cc6c31bf7cdfb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:55.528276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.530388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.532556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1016/2032] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:55 2022] DELETE /identity/v3/projects/243a2af6952540ac8cc6c31bf7cdfb8e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:55.538295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a46f88-744a-46e2-bc96-7ae9803bd9e3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.538653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a46f88-744a-46e2-bc96-7ae9803bd9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.538939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a46f88-744a-46e2-bc96-7ae9803bd9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:55.590340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-26a46f88-744a-46e2-bc96-7ae9803bd9e3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:55.592560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-26a46f88-744a-46e2-bc96-7ae9803bd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:55.632679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:55.670413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:55.672427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:55.683033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:55.739478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1017/2033] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:55.741188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7828dc-9201-497d-b940-a45f8ec5bf24 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.741624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7828dc-9201-497d-b940-a45f8ec5bf24 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.742061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7828dc-9201-497d-b940-a45f8ec5bf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:55.767489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4c7828dc-9201-497d-b940-a45f8ec5bf24 None None] MFA Rules not processed for user `c969fad4feb7468abd62714899d3fc2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:55.769347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4c7828dc-9201-497d-b940-a45f8ec5bf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:55.802215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fe23-c7fe-4fe7-9f98-c363da703e4c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:55.818995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:55.820371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.821393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.821534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] PATH_INFO: `/v3/projects/3d45ba62e84f43dfb0e9083a793a1112` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:55.823287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d45ba62e84f43dfb0e9083a793a1112)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:55.826986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:55.859604 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1018/2034] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:55.861015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.861307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.861586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:55.912667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d45ba62e84f43dfb0e9083a793a1112', 'request_id': 'req-e31ec1e3-1437-4aa9-984c-8f332f51953b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.915306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:55.921342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:55.921575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d45ba62e84f43dfb0e9083a793a1112', 'request_id': 'req-e31ec1e3-1437-4aa9-984c-8f332f51953b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.934626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d45ba62e84f43dfb0e9083a793a1112', 'request_id': 'req-e31ec1e3-1437-4aa9-984c-8f332f51953b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.957679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] The token cache is being invalidate because project 3d45ba62e84f43dfb0e9083a793a1112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:33:55.957970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.958736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31ec1e3-1437-4aa9-984c-8f332f51953b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:33:55.962489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1017/2035] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:33:55 2022] DELETE /identity/v3/projects/3d45ba62e84f43dfb0e9083a793a1112 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:55.968865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebe95a-6d29-4466-94d5-5c67c8b0079d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:55.969648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebe95a-6d29-4466-94d5-5c67c8b0079d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:55.971040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebe95a-6d29-4466-94d5-5c67c8b0079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.024998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-eaebe95a-6d29-4466-94d5-5c67c8b0079d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:56.028334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-eaebe95a-6d29-4466-94d5-5c67c8b0079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:56.053342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1019/2036] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:56.058689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.078896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d50d12-27ea-47fd-89cd-23d9b476630f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.092119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.093033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.093417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.093785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.094632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:56.096288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.100429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.129991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1020/2037] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:56.137886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.158766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1018/2038] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:56.168562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.168825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.169558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.184415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889eb76-e001-4273-8bdc-87602dc5e8c9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.208891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.209581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.209863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.210136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.210838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.220461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.232014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:56.237472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:56.267205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1019/2039] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:56.272218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.286325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadd891-7760-4ee0-ad69-37e0f6c83abd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.291451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1021/2040] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:56 2022] POST /identity/v3/projects => generated 399 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:56.298319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.300220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.301345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.301713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.302081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.302943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:33:56.303340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.306133 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.349935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1020/2041] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:33:56.351695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.368083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea350-04a5-4b8b-974b-2f4b9f64e916 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.372200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:56.397209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:56.440987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6aac8-9c95-455f-9e3c-a7950e8bfd3d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.453484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.454040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.454324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.454598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.456033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.457908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.504759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1022/2042] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:56 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:33:56.507177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.524960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d626e6-5708-4455-af22-61f1e6923e80 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.530112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.531245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.531428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.531785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.532116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.532229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.532397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.532698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.533441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.533578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.535069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.536403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.564951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1023/2043] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:56.569271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.586757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1021/2044] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:56 2022] POST /identity/v3/projects => generated 393 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:56.588721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.594491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e39dbb-41ce-45fe-b878-8a4d0b1f544c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.740874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.741639 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.741923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.742198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.742951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.744576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.788680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1024/2045] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:56.791025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.824992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555c372-6e45-46a4-950b-c3cf071ab60a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.833170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54c00c-d5a4-4b31-b891-1afcd5c98a49 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.840757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.841711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.841711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.841885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.842581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.844542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.848864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596acc-7a0d-4393-9099-6fa57bd881de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:56.848864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596acc-7a0d-4393-9099-6fa57bd881de admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.849200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596acc-7a0d-4393-9099-6fa57bd881de admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.852320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596acc-7a0d-4393-9099-6fa57bd881de admin admin] PATH_INFO: `/v3/projects/22482354aa87453183c1285be4163e76/users/18085978a3344066a91dccb14ceaf212/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:56.897699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596acc-7a0d-4393-9099-6fa57bd881de admin admin] RBAC: Authorizing `identity:create_grant(project_id=22482354aa87453183c1285be4163e76, user_id=18085978a3344066a91dccb14ceaf212, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:56.900378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596acc-7a0d-4393-9099-6fa57bd881de admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:56.907193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1022/2046] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:56 2022] POST /identity/v3/users => generated 297 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:56.911607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:56.932336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6568f1-3701-45b0-8cef-8a35b6524548 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:56.932722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1025/2047] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:56 2022] PUT /identity/v3/projects/22482354aa87453183c1285be4163e76/users/18085978a3344066a91dccb14ceaf212/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:56.943744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:56.944232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:56.944727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.023354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] MFA Rules not processed for user `18085978a3344066a91dccb14ceaf212`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:57.028927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:57.103256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.103939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.104219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.104488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.105251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.107061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.137638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1023/2048] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:57.140812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.150017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1026/2049] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:57.150958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.161197 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9ff83-626b-4cbf-b1a4-8d710bd5cfed admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.176498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.177630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.177903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf48d7-9736-4d91-acac-229f63fa2559 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.178390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.178866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.179914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.182187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.190526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.191211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.191492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.191763 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.192466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.195707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.208779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:57.212293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1024/2050] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:57.213564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.225618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069466-a01b-4494-a31f-f201ac14b48e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.240036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f68e0-e86e-4bbd-9880-691c22aa5bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.246127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1027/2051] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:57 2022] POST /identity/v3/projects => generated 401 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:57.246731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f68e0-e86e-4bbd-9880-691c22aa5bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.247415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f68e0-e86e-4bbd-9880-691c22aa5bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.247874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.248006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f68e0-e86e-4bbd-9880-691c22aa5bf9 admin admin] PATH_INFO: `/v3/projects/5482c3daeb06409b8ba6ec231b0b6050/users/050ccd6efbc84a1db655a2e9ed7f347d/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.316938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:57.331509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f68e0-e86e-4bbd-9880-691c22aa5bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5482c3daeb06409b8ba6ec231b0b6050, user_id=050ccd6efbc84a1db655a2e9ed7f347d, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.343724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f68e0-e86e-4bbd-9880-691c22aa5bf9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.387956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1025/2052] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:57 2022] PUT /identity/v3/projects/5482c3daeb06409b8ba6ec231b0b6050/users/050ccd6efbc84a1db655a2e9ed7f347d/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:57.400661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.401009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.401289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.435185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18db690-5207-493f-9d8d-8add7bf1f2a4 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.503243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] MFA Rules not processed for user `050ccd6efbc84a1db655a2e9ed7f347d`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:57.504504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:57.514455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.515168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.515519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.516205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.517032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.520452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.554507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1028/2053] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:57 2022] POST /identity/v3/users => generated 307 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:57.559177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.582256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377170c-053f-4d42-8f7b-deb35e6069b4 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.596198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.597274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.598018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.599892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.601532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.607509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.634701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1029/2054] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:57.643073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.666836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512de0d2-5e6f-47ed-8f9d-85ed2ce5612c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.671548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1026/2055] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:57.678030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.690055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.702477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6d510-a7c9-4f2a-9f58-bfb6f62beba0 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.704840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.705497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.706007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] PATH_INFO: `/v3/projects/d6f91bcbaa584a49b7988045b95447f5/users/3d87f6ae9feb40939dd04e492e39563c/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.707748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.708529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.708938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.709307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.710145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.712236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.727428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f91bcbaa584a49b7988045b95447f5, user_id=3d87f6ae9feb40939dd04e492e39563c, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.730839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.750155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1027/2056] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:33:57 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:57.754347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.771307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1030/2057] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:57 2022] PUT /identity/v3/projects/d6f91bcbaa584a49b7988045b95447f5/users/3d87f6ae9feb40939dd04e492e39563c/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:57.773025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.822750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363ad8d-a72b-4da4-82bf-b5b4e38ca476 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.827953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.828683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.828965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.829240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.830120 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.832072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.879453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1028/2058] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:33:57 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:33:57.880876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.892092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:57.893750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9c327-e3b5-4f72-9287-c2ffa3f39c07 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.900996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.901714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.902022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.902301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.903256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.905103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.923162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea3aa2-2f27-4634-9572-772502352e93 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.932775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.933574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.935571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.935929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.936804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.941159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1029/2059] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:57.941807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.945866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.958023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c09d52-4b93-4353-9054-70062f0862a3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:57.964003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1031/2060] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:33:57.966476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:57.967588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:57.968000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:57.968417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] PATH_INFO: `/v3/projects/da09045b498d4b37aa0ddfe8faec7e47/users/9655aa73803c48fcb9bf1a6b997f4055/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:57.968794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:57.988769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da09045b498d4b37aa0ddfe8faec7e47, user_id=9655aa73803c48fcb9bf1a6b997f4055, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:57.991960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:57.999030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd8507-31c7-4d26-bd20-72ed5a61eeeb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.004776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f05f46-02be-45b9-af2b-ef64cd1013cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.005778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05f46-02be-45b9-af2b-ef64cd1013cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.006146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05f46-02be-45b9-af2b-ef64cd1013cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.006507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f05f46-02be-45b9-af2b-ef64cd1013cc admin admin] PATH_INFO: `/v3/projects/d6f91bcbaa584a49b7988045b95447f5/users/3d87f6ae9feb40939dd04e492e39563c/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.008764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f05f46-02be-45b9-af2b-ef64cd1013cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f91bcbaa584a49b7988045b95447f5, user_id=3d87f6ae9feb40939dd04e492e39563c, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.012218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f05f46-02be-45b9-af2b-ef64cd1013cc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.017179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1030/2061] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:57 2022] PUT /identity/v3/projects/da09045b498d4b37aa0ddfe8faec7e47/users/9655aa73803c48fcb9bf1a6b997f4055/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:33:58.018541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:58.038983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1032/2062] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:57 2022] PUT /identity/v3/projects/d6f91bcbaa584a49b7988045b95447f5/users/3d87f6ae9feb40939dd04e492e39563c/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:58.041082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:58.052003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.052293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.052561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.088275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] MFA Rules not processed for user `3d87f6ae9feb40939dd04e492e39563c`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:58.089815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:58.109201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe52ff-f890-4239-b752-76b6d7577af7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.167081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1033/2063] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:58.174532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.175270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.175561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.175981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.176761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.178424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.180007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:58.181947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:58.192681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1031/2064] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:33:58.195305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:58.206419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13da32-fdef-446c-bb6e-bbaed6d0c927 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.211727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b805-a231-486c-b9e0-8e7aa6417ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.212667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b805-a231-486c-b9e0-8e7aa6417ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.213089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b805-a231-486c-b9e0-8e7aa6417ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.213365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b805-a231-486c-b9e0-8e7aa6417ba3 admin admin] PATH_INFO: `/v3/projects/da09045b498d4b37aa0ddfe8faec7e47/users/9655aa73803c48fcb9bf1a6b997f4055/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.215612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b805-a231-486c-b9e0-8e7aa6417ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da09045b498d4b37aa0ddfe8faec7e47, user_id=9655aa73803c48fcb9bf1a6b997f4055, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.218490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b805-a231-486c-b9e0-8e7aa6417ba3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.229654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1032/2065] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:33:58 2022] PUT /identity/v3/projects/da09045b498d4b37aa0ddfe8faec7e47/users/9655aa73803c48fcb9bf1a6b997f4055/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:33:58.242342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.242631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.242911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.244096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b08da-8fe2-4c43-9ff3-fca59a04797e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.274233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] MFA Rules not processed for user `9655aa73803c48fcb9bf1a6b997f4055`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:33:58.275436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:58.299200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.299889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.300174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.300446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.334367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:58.356896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1033/2066] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:33:58.366025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:58.368262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:33:58.371108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.372772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.384791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6380f3-0b48-444e-a008-5771edcf23b9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.388524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1034/2067] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:58.395092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.395869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.396044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.396304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.408672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.410587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.423962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1034/2068] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:58.491651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:58.501130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928c1f-db6a-431d-bd06-6516b69157f5 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.505893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:33:58.506077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.506835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.507184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.507565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.515517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2a7b8-f2a3-4591-8f16-b6caa67bad75 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:33:58.519597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:33:58.520278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:33:58.520556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:33:58.521005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:33:58.533038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:33:58.579862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.581917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.591132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:33:58.591660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1035/2069] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:33:58.592771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:33:58.602295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1035/2070] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:04.910118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:05.023224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593ffb-35f4-41d9-920e-c5fb43d3a9e7 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:05.030825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:05.031281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:05.031854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:05.032056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:05.032323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] PATH_INFO: `/v3/users/050ccd6efbc84a1db655a2e9ed7f347d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:05.033708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050ccd6efbc84a1db655a2e9ed7f347d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:05.036343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:05.047658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb730-79ed-42da-9504-8f9eacfb1cab service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:05.052857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:05.053832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:05.054115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:05.054388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] PATH_INFO: `/v3/users/18085978a3344066a91dccb14ceaf212` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:05.055640 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18085978a3344066a91dccb14ceaf212)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:05.066575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:05.123041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050ccd6efbc84a1db655a2e9ed7f347d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.123698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050ccd6efbc84a1db655a2e9ed7f347d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.138490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18085978a3344066a91dccb14ceaf212'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.139850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18085978a3344066a91dccb14ceaf212'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.144907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050ccd6efbc84a1db655a2e9ed7f347d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.145844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050ccd6efbc84a1db655a2e9ed7f347d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.151699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:05.166265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe0533e0-1fe9-4c97-b368-f82269f4a813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050ccd6efbc84a1db655a2e9ed7f347d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.180680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18085978a3344066a91dccb14ceaf212'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.181517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18085978a3344066a91dccb14ceaf212'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.198156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-16c6df11-afd9-4dc1-82ce-0c9bbb760a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18085978a3344066a91dccb14ceaf212'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:05.215128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1036/2071] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:04 2022] DELETE /identity/v3/users/050ccd6efbc84a1db655a2e9ed7f347d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:05.220452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:05.220862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:05.221434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:05.224491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1036/2072] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:05 2022] DELETE /identity/v3/users/18085978a3344066a91dccb14ceaf212 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:05.229445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:05.229728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:05.229983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:05.250586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:05.252755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:05.312437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:05.314069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:05.375667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1037/2073] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:05.393035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:05.399407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:05.449683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1037/2074] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:05.454923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:05.456479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:05.491068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3025fb-3b30-41bd-b481-30ec81f8ed15 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:05.496337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:05.497087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:05.497373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:05.497652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:05.512219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:05.513809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:05.513950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:05.525876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1038/2075] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:05.556875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b0acc-3190-4c90-9611-fd5f7e0fe665 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:05.561956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:05.563032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:05.563146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:05.563449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:05.574147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:05.576593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:05.589498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1038/2076] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:06.001610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:06.003231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:06.034396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ba054-ce42-4abf-9593-8a810142e0c0 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:06.043422 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:06.044543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:06.044977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:06.045202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] PATH_INFO: `/v3/projects/5482c3daeb06409b8ba6ec231b0b6050` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:06.055236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:06.057940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5482c3daeb06409b8ba6ec231b0b6050)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:06.058137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:06.063623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:06.080786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c43f42-b686-411e-b775-406e522ae43e service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:06.087076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:06.088565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:06.089272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:06.090213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] PATH_INFO: `/v3/projects/22482354aa87453183c1285be4163e76` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:06.091794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22482354aa87453183c1285be4163e76)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:06.095214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:06.151245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5482c3daeb06409b8ba6ec231b0b6050', 'request_id': 'req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.162667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22482354aa87453183c1285be4163e76', 'request_id': 'req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.163165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5482c3daeb06409b8ba6ec231b0b6050', 'request_id': 'req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.171921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22482354aa87453183c1285be4163e76', 'request_id': 'req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.180160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22482354aa87453183c1285be4163e76', 'request_id': 'req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.182030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5482c3daeb06409b8ba6ec231b0b6050', 'request_id': 'req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.198354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] The token cache is being invalidate because project 5482c3daeb06409b8ba6ec231b0b6050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:06.198875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.199661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.202143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] The token cache is being invalidate because project 22482354aa87453183c1285be4163e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:06.202584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.203231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.205007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1039/2078] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:06 2022] DELETE /identity/v3/projects/22482354aa87453183c1285be4163e76 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:06.205632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1039/2078] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:06 2022] DELETE /identity/v3/projects/5482c3daeb06409b8ba6ec231b0b6050 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:06.206223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:06.207626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:06.209218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:06.210826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:06.224894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:06.287439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051325d-e7c5-4ca4-9ddb-0ae609902f3b admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:06.288605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad0401-4a4c-473e-b34e-edd9b5ecac78 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:06.293474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:06.294380 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:06.294686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:06.294934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] PATH_INFO: `/v3/users/3d87f6ae9feb40939dd04e492e39563c` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:06.296127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d87f6ae9feb40939dd04e492e39563c)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:06.299267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:06.299959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:06.300916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:06.301161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:06.301439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] PATH_INFO: `/v3/users/9655aa73803c48fcb9bf1a6b997f4055` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:06.305776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9655aa73803c48fcb9bf1a6b997f4055)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:06.310277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:06.384592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d87f6ae9feb40939dd04e492e39563c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.385310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d87f6ae9feb40939dd04e492e39563c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.396962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9655aa73803c48fcb9bf1a6b997f4055'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.397348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9655aa73803c48fcb9bf1a6b997f4055'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.403344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:06.416767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d87f6ae9feb40939dd04e492e39563c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.417480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d87f6ae9feb40939dd04e492e39563c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.422899 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9655aa73803c48fcb9bf1a6b997f4055'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.423700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9655aa73803c48fcb9bf1a6b997f4055'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.429669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d87f6ae9feb40939dd04e492e39563c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.435294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9655aa73803c48fcb9bf1a6b997f4055'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:06.453877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1040/2079] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:06 2022] DELETE /identity/v3/users/3d87f6ae9feb40939dd04e492e39563c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:06.455965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1040/2080] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:06 2022] DELETE /identity/v3/users/9655aa73803c48fcb9bf1a6b997f4055 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:06.938188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:06.939730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:06.945273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:06.948255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.040113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e479d6-03fa-4e29-843f-2f8805f68bd1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.047156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.048994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.049279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.049558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] PATH_INFO: `/v3/projects/da09045b498d4b37aa0ddfe8faec7e47` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.051004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da09045b498d4b37aa0ddfe8faec7e47)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.054154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.055886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105506bc-d451-44d7-8b37-a19ea2c9df40 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.061519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.062949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.063233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.064062 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] PATH_INFO: `/v3/projects/d6f91bcbaa584a49b7988045b95447f5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.068042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f91bcbaa584a49b7988045b95447f5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.070901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.108137 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:07.129740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da09045b498d4b37aa0ddfe8faec7e47', 'request_id': 'req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.133217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f91bcbaa584a49b7988045b95447f5', 'request_id': 'req-540d5031-2071-441c-b291-fb3864b7e2e8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.140494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da09045b498d4b37aa0ddfe8faec7e47', 'request_id': 'req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.143904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f91bcbaa584a49b7988045b95447f5', 'request_id': 'req-540d5031-2071-441c-b291-fb3864b7e2e8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.150682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f91bcbaa584a49b7988045b95447f5', 'request_id': 'req-540d5031-2071-441c-b291-fb3864b7e2e8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.151761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da09045b498d4b37aa0ddfe8faec7e47', 'request_id': 'req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.165777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] The token cache is being invalidate because project d6f91bcbaa584a49b7988045b95447f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:07.166088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.169409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-540d5031-2071-441c-b291-fb3864b7e2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.170906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] The token cache is being invalidate because project da09045b498d4b37aa0ddfe8faec7e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:07.171821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.175564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1041/2081] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:06 2022] DELETE /identity/v3/projects/d6f91bcbaa584a49b7988045b95447f5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:07.176964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c5fb8be4-af8d-4ebc-b426-c6cf8ffb2a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.177931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.178371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.178713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.180509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1041/2082] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:06 2022] DELETE /identity/v3/projects/da09045b498d4b37aa0ddfe8faec7e47 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:07.185547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8662bd-653a-4345-9aa9-9028e8de6d04 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.186986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8662bd-653a-4345-9aa9-9028e8de6d04 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.187092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8662bd-653a-4345-9aa9-9028e8de6d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.200008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:07.201367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.211867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ba8662bd-653a-4345-9aa9-9028e8de6d04 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:07.213129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ba8662bd-653a-4345-9aa9-9028e8de6d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.274329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:07.286854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1042/2083] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:07.287920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.289313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.303123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e3ae-93b7-4f0b-88a1-770639e3acb6 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.307894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1042/2084] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:07.315627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfbf99-99d0-4731-8936-1d4267d72bb9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.315627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfbf99-99d0-4731-8936-1d4267d72bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.315627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfbf99-99d0-4731-8936-1d4267d72bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.315839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.315839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.315839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.315839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] PATH_INFO: `/v3/users/c3f271f137054858902257e22192907d` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.318887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f271f137054858902257e22192907d)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.322168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.329031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a7bfbf99-99d0-4731-8936-1d4267d72bb9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:07.330256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a7bfbf99-99d0-4731-8936-1d4267d72bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.338931 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1043/2085] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:07.340316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.340909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.340909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.359143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:07.360456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.369767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1044/2086] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:07.370368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.380569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bb01c-beae-4329-a69a-29a52770e6f4 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.385789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.386370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f271f137054858902257e22192907d'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.386484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.387249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.387362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f271f137054858902257e22192907d'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.387684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.388429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:07.388779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.390402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.402413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f271f137054858902257e22192907d'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.404675 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1045/2087] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:07.404810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f271f137054858902257e22192907d'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.404891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.407882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.413240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f271f137054858902257e22192907d'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:07.421181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f284-62e9-4133-a0d8-9bb2197b4496 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.426294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.426902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1043/2088] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:07 2022] DELETE /identity/v3/users/c3f271f137054858902257e22192907d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:07.427012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.427239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.427554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.427650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.428286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:07.428595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.428958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.430290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43df70-4df5-4612-9b60-663090826dec admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.443233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1046/2089] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:07.444516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.444829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.445106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.481656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:07.483242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.538651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058f7f5-a2ea-412f-8a82-586a9c4a2011 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.547132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.547817 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.548099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.548399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.549159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.552762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.586906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1047/2090] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:07.587791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.610807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b054c8-1ef9-4dbc-950b-6bd18373ca8d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.615914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1044/2091] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:07.615914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.622461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:07.648775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:07.706538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3be7-e2c7-4ac9-b65c-77f8822b35b7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.710715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.711386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.712018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.712297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.718593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.728041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.728041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.728041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.728041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.728041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.728646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.728646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.729529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:07.735826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1045/2092] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:07.737125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.749348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ed6aa-3ed9-4b37-b663-6fa93f90641a service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.754950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1048/2093] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:07.758908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.761126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.762271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.762631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.763199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.763921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.766707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.797264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1046/2094] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/users => generated 291 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:07.798859 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.812427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd7d69-7b34-46b2-95bb-4041b8d3022b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.851054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3384-d4b9-4562-8d3b-81fa820dcedb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.860205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.861083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.861472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.861807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.862572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.864886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.895936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1049/2095] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:07 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:07.901229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.922630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c6a30-3341-4f28-a0b5-b22690535245 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.928914 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.929072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.929718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.930133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.930272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.930428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.930667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.930776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.931409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.931543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.933631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.933746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.946058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1047/2096] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:07.950956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1050/2097] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:07.951212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.951888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.963308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8676ea1-95cc-4795-bae2-b1cd70aaa1c2 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.969665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500dda5-8e27-4608-94cc-fd073f37bf22 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:07.970985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.971828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.972934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.974456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.974456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.976585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.979296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:07.980138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:07.980614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:07.981481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:07.982429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:07.984110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:07.989447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1051/2098] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:07.993373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:07.995190 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1048/2099] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:08.000193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.004034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb9538-577e-487a-8523-ae3e104999bb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.009295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3c2d6-943c-4b47-97e0-adf2ad6fb6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.010263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3c2d6-943c-4b47-97e0-adf2ad6fb6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.010986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3c2d6-943c-4b47-97e0-adf2ad6fb6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.010986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3c2d6-943c-4b47-97e0-adf2ad6fb6c0 admin admin] PATH_INFO: `/v3/projects/4639c72e6c3b4757b490e506cc440a61/users/a48953eb50904969bacece6c5390501e/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.011891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848191d-b493-48ee-954f-03feef253d6a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.017142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b22e23-19cd-47ac-ad15-faa821caf654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.018108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b22e23-19cd-47ac-ad15-faa821caf654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.018419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b22e23-19cd-47ac-ad15-faa821caf654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.018700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b22e23-19cd-47ac-ad15-faa821caf654 admin admin] PATH_INFO: `/v3/projects/d6755255c60b4c04a3c4f0f884144ded/users/9e4470f4a2d74c07b78092a873a3a595/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.028369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3c2d6-943c-4b47-97e0-adf2ad6fb6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4639c72e6c3b4757b490e506cc440a61, user_id=a48953eb50904969bacece6c5390501e, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.031321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3c2d6-943c-4b47-97e0-adf2ad6fb6c0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.037299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b22e23-19cd-47ac-ad15-faa821caf654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6755255c60b4c04a3c4f0f884144ded, user_id=9e4470f4a2d74c07b78092a873a3a595, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.040934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b22e23-19cd-47ac-ad15-faa821caf654 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.052275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1052/2100] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:07 2022] PUT /identity/v3/projects/4639c72e6c3b4757b490e506cc440a61/users/a48953eb50904969bacece6c5390501e/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:08.061646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1049/2101] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:07 2022] PUT /identity/v3/projects/d6755255c60b4c04a3c4f0f884144ded/users/9e4470f4a2d74c07b78092a873a3a595/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:08.062837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.063518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.063518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.067789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.068076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.068351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.094953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] MFA Rules not processed for user `a48953eb50904969bacece6c5390501e`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:08.096215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.096750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] MFA Rules not processed for user `9e4470f4a2d74c07b78092a873a3a595`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:08.098086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.148168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:08.168905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1050/2102] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:08.173729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.177933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.180781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1053/2103] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:08.185643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.187099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.221425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:08.247548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebe99d-c1ea-43a7-99ef-fc89a36022d8 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.252340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.252784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d599eb-067c-47d9-a5e3-823e3d8d657f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.253033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.253313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.253588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.257558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.258425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.258712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.259008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] PATH_INFO: `/v3/projects/17ecb6953f7f4a689b4bb605832c5c6c` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.260219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ecb6953f7f4a689b4bb605832c5c6c)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.262313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.262500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.263802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.273336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1054/2104] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:08.278894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.290878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6054f8-5ac8-4dd9-8b4f-6acaf903fa0a service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.298107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.298766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.299045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.299316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.300001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.301980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.317975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ecb6953f7f4a689b4bb605832c5c6c', 'request_id': 'req-6afc5386-c2cb-4c22-a46e-9c821cf0adba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.325097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ecb6953f7f4a689b4bb605832c5c6c', 'request_id': 'req-6afc5386-c2cb-4c22-a46e-9c821cf0adba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.329612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1055/2105] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:08.331008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ecb6953f7f4a689b4bb605832c5c6c', 'request_id': 'req-6afc5386-c2cb-4c22-a46e-9c821cf0adba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.333481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.346655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] The token cache is being invalidate because project 17ecb6953f7f4a689b4bb605832c5c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:08.346951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.347723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.349359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1051/2106] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:08 2022] DELETE /identity/v3/projects/17ecb6953f7f4a689b4bb605832c5c6c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:08.350099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.351435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.404743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b000ff-2134-43a8-aee6-232d18db533a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.409772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.410666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.410946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.411357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] PATH_INFO: `/v3/users/a48953eb50904969bacece6c5390501e` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.412590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48953eb50904969bacece6c5390501e)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.414560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.427805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc5386-c2cb-4c22-a46e-9c821cf0adba admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.433004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cedbc5-0a66-4888-a46c-312582be66b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.433810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cedbc5-0a66-4888-a46c-312582be66b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.434089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cedbc5-0a66-4888-a46c-312582be66b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.434489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cedbc5-0a66-4888-a46c-312582be66b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.435196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cedbc5-0a66-4888-a46c-312582be66b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.437243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cedbc5-0a66-4888-a46c-312582be66b0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.460964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1052/2107] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:08.460964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547f920-b960-4906-9cd9-45953c791add None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.460964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547f920-b960-4906-9cd9-45953c791add None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.460964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547f920-b960-4906-9cd9-45953c791add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.474248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48953eb50904969bacece6c5390501e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.475048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48953eb50904969bacece6c5390501e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.481553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7547f920-b960-4906-9cd9-45953c791add None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:08.482571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7547f920-b960-4906-9cd9-45953c791add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.490954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:08.491431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1053/2108] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:08.492272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547f920-b960-4906-9cd9-45953c791add None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.494100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7547f920-b960-4906-9cd9-45953c791add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.497230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48953eb50904969bacece6c5390501e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.498406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48953eb50904969bacece6c5390501e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.509669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547f920-b960-4906-9cd9-45953c791add None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.510193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9c4589f0-3c6f-42be-ab57-c22856f7e2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48953eb50904969bacece6c5390501e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:08.519174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67569c6f-b72f-4a44-932b-de0ee04cf764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.520258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67569c6f-b72f-4a44-932b-de0ee04cf764 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.520539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67569c6f-b72f-4a44-932b-de0ee04cf764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.520840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67569c6f-b72f-4a44-932b-de0ee04cf764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.521587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67569c6f-b72f-4a44-932b-de0ee04cf764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.523232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67569c6f-b72f-4a44-932b-de0ee04cf764 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.523994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1056/2109] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:08 2022] DELETE /identity/v3/users/a48953eb50904969bacece6c5390501e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:08.525302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.525588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.525872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.536471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1054/2110] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:08.537779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.538066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.538332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.544025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:08.545278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.556491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:08.563648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.595747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:08.609485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1057/2111] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:08.610313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.619386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d98d6f-6d6e-46a5-b2ee-8d9c1d2a7e97 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.624828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.625766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.626049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.626320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] PATH_INFO: `/v3/projects/7fab89566e904f5bbb73bca55448f9e0/users/f61108cd1d834fd6803fc7021669268b/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.636413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1055/2112] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:08.636413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.641886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fab89566e904f5bbb73bca55448f9e0, user_id=f61108cd1d834fd6803fc7021669268b, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.645200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.652689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cce49-9cc4-43f5-b120-831ba37bac97 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.658712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.659559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.659984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.660432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.661000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1058/2113] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:08 2022] PUT /identity/v3/projects/7fab89566e904f5bbb73bca55448f9e0/users/f61108cd1d834fd6803fc7021669268b/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:08.661514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:08.661837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.662121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.663506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:08.664071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.680669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1056/2114] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:08.680669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.689566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ee13a-4c97-47b2-b83c-6238b40c31c3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.725048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcda51a-032f-45a9-95bf-c2ff7a1025a9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.728607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.731536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.731536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.731536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.748048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.748834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.749053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.749741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.750478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.752367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.764119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1057/2115] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:08.765039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.775090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3147c-0ca4-4bb1-b8c0-0a5a4f25dd14 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.780670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.781340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.781707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.781887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.782568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.784614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.799702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.801212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.811939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1059/2116] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:08.812860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.817308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1058/2117] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:08.818574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.824980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc25c4-08ef-4709-9447-4d854b92099c service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.830905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:08.908017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf8e37-5bdc-4c32-97cc-6a0657299cf5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:08.926862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.927556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.927833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.928187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.928943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.932488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69d2c9-bd78-4d56-879d-c32402468ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:08.933992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69d2c9-bd78-4d56-879d-c32402468ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.934629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69d2c9-bd78-4d56-879d-c32402468ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.935119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69d2c9-bd78-4d56-879d-c32402468ba7 admin admin] PATH_INFO: `/v3/projects/7fab89566e904f5bbb73bca55448f9e0/users/f61108cd1d834fd6803fc7021669268b/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.937616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69d2c9-bd78-4d56-879d-c32402468ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fab89566e904f5bbb73bca55448f9e0, user_id=f61108cd1d834fd6803fc7021669268b, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:08.940526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69d2c9-bd78-4d56-879d-c32402468ba7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.940988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:08.956281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1060/2118] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:08 2022] PUT /identity/v3/projects/7fab89566e904f5bbb73bca55448f9e0/users/f61108cd1d834fd6803fc7021669268b/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:08.969693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:08.969693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:08.969955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:08.980163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1059/2119] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:08.984515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:08.998951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7084-9f19-41ba-aae3-bea95d98f364 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.002319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] MFA Rules not processed for user `f61108cd1d834fd6803fc7021669268b`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:09.003721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.041590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:09.088892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1061/2120] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:09.094781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.096464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.097452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.097886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.098965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.100787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.101335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.102753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.115093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1060/2121] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:09.118639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.137102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a231-d76f-4cb2-9ad0-2f879be7135c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.145852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.149029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.149304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.150361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.151087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.153473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.170951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1061/2122] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:09.175649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.191268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee267fc-a413-42bd-9163-c144e1fb4930 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.193380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db13af-29df-4fb1-b991-da0ae423890f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.200064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.200775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.201079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.201365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.202710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1955d5-a311-4ed8-a334-01acdd46d953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.203662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1955d5-a311-4ed8-a334-01acdd46d953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.203946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1955d5-a311-4ed8-a334-01acdd46d953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.204261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1955d5-a311-4ed8-a334-01acdd46d953 admin admin] PATH_INFO: `/v3/projects/6bcaed2b380f44bc82c7537fa4aca464/users/b1c98bf98b644312843a305486fb9f1a/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.222134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.223984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.236663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1955d5-a311-4ed8-a334-01acdd46d953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcaed2b380f44bc82c7537fa4aca464, user_id=b1c98bf98b644312843a305486fb9f1a, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.238760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1955d5-a311-4ed8-a334-01acdd46d953 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.246350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1062/2123] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:09.247256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.249065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.267015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ee9e7-2834-449b-8a13-f74e7517bdfb service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.275071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1062/2124] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:09 2022] PUT /identity/v3/projects/6bcaed2b380f44bc82c7537fa4aca464/users/b1c98bf98b644312843a305486fb9f1a/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:09.292689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.293288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.293687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.305786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:09.336870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] MFA Rules not processed for user `b1c98bf98b644312843a305486fb9f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:09.338126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.381522 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.383891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.385662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.385951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] PATH_INFO: `/v3/projects/4639c72e6c3b4757b490e506cc440a61` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.388198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4639c72e6c3b4757b490e506cc440a61)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.390326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.450043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1063/2125] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:09.455957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.457733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.462974 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4639c72e6c3b4757b490e506cc440a61', 'request_id': 'req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.479587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4639c72e6c3b4757b490e506cc440a61', 'request_id': 'req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.488176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4639c72e6c3b4757b490e506cc440a61', 'request_id': 'req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.500916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] The token cache is being invalidate because project 4639c72e6c3b4757b490e506cc440a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:09.501178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.501881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bd35095c-b6e8-46e0-8c8f-1cd8e38d6671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.503454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1063/2126] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:09 2022] DELETE /identity/v3/projects/4639c72e6c3b4757b490e506cc440a61 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:09.508565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f505ea-a8c4-4b23-bb24-75b385579aa2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.508909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f505ea-a8c4-4b23-bb24-75b385579aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.509770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f505ea-a8c4-4b23-bb24-75b385579aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.533794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-49f505ea-a8c4-4b23-bb24-75b385579aa2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:09.534621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-49f505ea-a8c4-4b23-bb24-75b385579aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.539913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8737f-4124-4740-840c-9cfe1d9b5c4a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.587565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:09.616103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.616810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.617097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.617423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.619339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.624453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1064/2127] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:09.632154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.632444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.632750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.666287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:09.667867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.677264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1065/2128] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:09.681977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.700903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4ab42-ae4e-49ba-8639-0ec87cac20c7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.708600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.709419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.712652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.713659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.714129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.715275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.717054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:09.717701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.719757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.724910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1064/2129] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:09.733704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1066/2130] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:34:09.738394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.749962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6bd88-0a43-406e-9645-9b37efc1e52d service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.754858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.756194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.756474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.756797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.757492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.759420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.783181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.785144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.797651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1065/2131] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:09 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:09.802109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.804191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895135-1e98-482a-adcb-03bea661ca21 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.853682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:09.871108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab11ee-34f5-4c3a-a87a-b64a2ee25be5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.877960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.878685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.878966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.879360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.880102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.880211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.881030 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.881313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.881585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] PATH_INFO: `/v3/users/9e4470f4a2d74c07b78092a873a3a595` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.882287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.882933 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4470f4a2d74c07b78092a873a3a595)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.887559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.908692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1066/2132] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:09 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:09.912792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.925080 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8568d-9ad4-4d2f-bae0-7d4b5211beec admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.930088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.930873 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.931253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.931648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.932473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.934520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:09.943260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4470f4a2d74c07b78092a873a3a595'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.944119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4470f4a2d74c07b78092a873a3a595'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.946926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1067/2133] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:09.950675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:09.954455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:09.974395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4470f4a2d74c07b78092a873a3a595'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.976066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4470f4a2d74c07b78092a873a3a595'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.978766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54032025-c2db-4ef5-93b1-8fe6a3c427a8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:09.985682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:09.986804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:09.987256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:09.987439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fd73be8d-2865-4fb2-b3b5-d948ad566073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4470f4a2d74c07b78092a873a3a595'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:09.987753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:09.988685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:09.991339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:10.004059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1067/2134] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:09 2022] DELETE /identity/v3/users/9e4470f4a2d74c07b78092a873a3a595 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:10.007068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1068/2135] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:10.008124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.008453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.008726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.011531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:10.026405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5c0d0-55ef-4984-aaa2-818da70b3cd4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:10.027955 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:10.029222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:10.077196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:10.101118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1068/2136] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:10.113056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:10.113928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbac901-b6b7-4db9-955c-ecc7f38a88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:10.114978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbac901-b6b7-4db9-955c-ecc7f38a88e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.115349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbac901-b6b7-4db9-955c-ecc7f38a88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.115808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbac901-b6b7-4db9-955c-ecc7f38a88e8 admin admin] PATH_INFO: `/v3/projects/79f02ad55da84c41834fd71c04d576ab/users/67fafa178edd4acf8da8d67bb6b24761/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.115975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:10.133562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbac901-b6b7-4db9-955c-ecc7f38a88e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f02ad55da84c41834fd71c04d576ab, user_id=67fafa178edd4acf8da8d67bb6b24761, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:10.137828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbac901-b6b7-4db9-955c-ecc7f38a88e8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:10.160815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1069/2137] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:10 2022] PUT /identity/v3/projects/79f02ad55da84c41834fd71c04d576ab/users/67fafa178edd4acf8da8d67bb6b24761/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:10.165931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.166299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.166621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.200342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] MFA Rules not processed for user `67fafa178edd4acf8da8d67bb6b24761`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:10.201615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:10.204751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373ad88-6fcf-4f4e-be8e-23656cc01a50 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:10.222012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:10.269192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:10.269996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.270556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.271126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.289851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1070/2138] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:10.306536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:10.308732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:10.342766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:10.344270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:10.354340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1069/2139] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:10.373952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2331dd2-63b8-4bb7-ae20-5888230082b7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:10.377644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:10.378306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.378587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.378861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.388257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:10.390134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:10.400393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1071/2140] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:10.697725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:10.700614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:10.725374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1feb9-2edd-49b2-a8bd-f2ab7554378b service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:10.743322 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:10.744257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.744619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.745437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] PATH_INFO: `/v3/projects/d6755255c60b4c04a3c4f0f884144ded` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.746864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6755255c60b4c04a3c4f0f884144ded)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:10.748939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:10.855646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6755255c60b4c04a3c4f0f884144ded', 'request_id': 'req-d6eb4146-d19c-4b2b-a580-7a7752a4966c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:10.863383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6755255c60b4c04a3c4f0f884144ded', 'request_id': 'req-d6eb4146-d19c-4b2b-a580-7a7752a4966c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:10.871428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6755255c60b4c04a3c4f0f884144ded', 'request_id': 'req-d6eb4146-d19c-4b2b-a580-7a7752a4966c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:10.885682 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] The token cache is being invalidate because project d6755255c60b4c04a3c4f0f884144ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:10.885979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:10.886735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:10.888576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1070/2141] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:10 2022] DELETE /identity/v3/projects/d6755255c60b4c04a3c4f0f884144ded => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:10.892350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:10.903448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caca5f8-d06f-4942-9a90-1975e84c944c service glance] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:10.957962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:10.973467 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:10.975347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:10.975659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:10.975896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] PATH_INFO: `/v3/users/f61108cd1d834fd6803fc7021669268b` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:10.977392 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61108cd1d834fd6803fc7021669268b)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:10.979455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.052767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61108cd1d834fd6803fc7021669268b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.053447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61108cd1d834fd6803fc7021669268b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.069181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61108cd1d834fd6803fc7021669268b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.069903 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61108cd1d834fd6803fc7021669268b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.078155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-5a298a12-65c9-40fb-ad6d-37bad7269c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61108cd1d834fd6803fc7021669268b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.092101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1072/2142] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:10 2022] DELETE /identity/v3/users/f61108cd1d834fd6803fc7021669268b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:11.438310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:11.439864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:11.454371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb4146-d19c-4b2b-a580-7a7752a4966c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:11.503197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:11.516282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:11.517215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.517500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.517782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] PATH_INFO: `/v3/projects/7fab89566e904f5bbb73bca55448f9e0` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.519405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fab89566e904f5bbb73bca55448f9e0)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:11.521758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.578431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fab89566e904f5bbb73bca55448f9e0', 'request_id': 'req-51d37d87-4dea-465c-899c-f1ecff9b896f'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.586271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fab89566e904f5bbb73bca55448f9e0', 'request_id': 'req-51d37d87-4dea-465c-899c-f1ecff9b896f'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.592318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fab89566e904f5bbb73bca55448f9e0', 'request_id': 'req-51d37d87-4dea-465c-899c-f1ecff9b896f'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.603357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] The token cache is being invalidate because project 7fab89566e904f5bbb73bca55448f9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:11.603642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.604267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-51d37d87-4dea-465c-899c-f1ecff9b896f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:11.606073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1071/2143] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:11 2022] DELETE /identity/v3/projects/7fab89566e904f5bbb73bca55448f9e0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:11.611235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.611740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.611740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.630798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:11.632029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:11.694945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1073/2144] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:11.706863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.707149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.707421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.728359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:11.729610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:11.738612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1072/2145] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:11.743005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:11.753317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc460480-0314-4e8b-877c-f2c4b5ed52dc None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:11.758445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:11.759104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.759410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.759682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.760415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:11.760748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:11.762404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.774409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1074/2146] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:11.779270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:11.788580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e416f-8e22-420f-944d-0f001d7c5fd2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:11.793670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:11.794759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.794759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.794918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.795562 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:11.797843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.825626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1073/2147] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:11 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:11.829723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:11.839947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afc930-5fc1-43bd-9692-f1c69449682f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:11.892725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:11.900568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:11.902287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.902693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.903094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.904088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:11.906499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.927493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1075/2148] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:11 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:11.931806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:11.942216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b4cca-ef35-4857-a6b3-50ab462e615f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:11.946911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:11.947575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.947854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.948126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.948907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:11.950521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.960831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1074/2149] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:11.965325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:11.975296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab866b6a-ee1b-411a-a60b-6842ffef47d7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:11.980080 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a554b4-a47f-4c65-beb7-9b6dffe4c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:11.980828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a554b4-a47f-4c65-beb7-9b6dffe4c87e admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:11.981127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a554b4-a47f-4c65-beb7-9b6dffe4c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:11.981398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a554b4-a47f-4c65-beb7-9b6dffe4c87e admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:11.982105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a554b4-a47f-4c65-beb7-9b6dffe4c87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:11.983716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a554b4-a47f-4c65-beb7-9b6dffe4c87e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:11.993882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1076/2150] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:11.998673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.013284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c33b2-a32e-4843-a009-3a56f91f5ac0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.019189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.020083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.020348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.020620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] PATH_INFO: `/v3/projects/f572e3eb5a0247f39dc2c225e52c0870/users/03265311bba341488aa155a689ce56e7/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.037758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f572e3eb5a0247f39dc2c225e52c0870, user_id=03265311bba341488aa155a689ce56e7, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.040815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.071188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1075/2151] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:11 2022] PUT /identity/v3/projects/f572e3eb5a0247f39dc2c225e52c0870/users/03265311bba341488aa155a689ce56e7/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:12.073649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.073949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.074284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.102737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] MFA Rules not processed for user `03265311bba341488aa155a689ce56e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:12.104112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:12.169199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1077/2152] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:12.179878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.241265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc620e-52f8-4735-bfc2-957fac95daeb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.246652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.247338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.247617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.247890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.248572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.250520 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.274157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:12.283011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1076/2153] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:12 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:12.287151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.298995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8166d-bc28-4a0f-8800-3a06fb9b71d5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.356543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.357336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.357616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.358555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.359288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.361283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.384978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1078/2154] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:12 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:12.389383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.400318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4a945-a57e-4186-b532-b779c7d5237f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.405524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.406201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.406484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.406762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.407497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.409163 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.420337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1077/2155] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:12.425105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.436109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061d906-cee1-4432-8e27-919b54eaa13f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.441798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.442823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.443104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.443375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] PATH_INFO: `/v3/projects/223bfff30b3c44c79261bc3aedcfd211/users/09afd78fd78c4d13b6aab3c43dba7613/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.461881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223bfff30b3c44c79261bc3aedcfd211, user_id=09afd78fd78c4d13b6aab3c43dba7613, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.464557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.480396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1079/2156] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:12 2022] PUT /identity/v3/projects/223bfff30b3c44c79261bc3aedcfd211/users/09afd78fd78c4d13b6aab3c43dba7613/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:12.484083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.494590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf0f36-07d4-436f-be48-9c435fe04410 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.553437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ffe9e-0acb-437f-91c8-54b57510a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.554053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ffe9e-0acb-437f-91c8-54b57510a00d admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.554336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ffe9e-0acb-437f-91c8-54b57510a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.554611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ffe9e-0acb-437f-91c8-54b57510a00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.555346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ffe9e-0acb-437f-91c8-54b57510a00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.557010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ffe9e-0acb-437f-91c8-54b57510a00d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.567123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1078/2157] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:12.571148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.582528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65e56-9414-4cbe-a160-411a949aaaa9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.587729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.588704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.588989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.589266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] PATH_INFO: `/v3/projects/223bfff30b3c44c79261bc3aedcfd211/users/09afd78fd78c4d13b6aab3c43dba7613/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.591386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223bfff30b3c44c79261bc3aedcfd211, user_id=09afd78fd78c4d13b6aab3c43dba7613, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.594123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.605386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1080/2158] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:12 2022] PUT /identity/v3/projects/223bfff30b3c44c79261bc3aedcfd211/users/09afd78fd78c4d13b6aab3c43dba7613/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:12.615864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.616158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.616440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.643994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] MFA Rules not processed for user `09afd78fd78c4d13b6aab3c43dba7613`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:12.645253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:12.733918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1079/2159] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:12.747871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:12.749883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:12.770705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:12.813203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a1018-6838-4c3f-bbb3-f98a5855d558 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:12.817060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:12.817816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:12.818093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:12.818398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:12.878767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:12.880207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:12.889325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1081/2160] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:20.038310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:20.039891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.107986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e0019-bbcb-4fe2-a91c-fc710cfa64bc None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:20.113099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:20.113983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:20.114261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:20.114540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] PATH_INFO: `/v3/users/8edd30cd3c384935b6f8c64734fc9ba3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:20.115767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8edd30cd3c384935b6f8c64734fc9ba3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:20.117749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:20.130578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:20.175984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8edd30cd3c384935b6f8c64734fc9ba3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.177220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8edd30cd3c384935b6f8c64734fc9ba3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.195176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8edd30cd3c384935b6f8c64734fc9ba3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.196144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8edd30cd3c384935b6f8c64734fc9ba3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.213054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eddf23fe-01d5-4d04-840b-8a4e9039c286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8edd30cd3c384935b6f8c64734fc9ba3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.213869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:20.223018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.227506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1080/2161] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:20 2022] DELETE /identity/v3/users/8edd30cd3c384935b6f8c64734fc9ba3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:20.232073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:20.232367 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:20.232824 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:20.244770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b657-1aec-4881-b35c-b243abdedfe5 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:20.256787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:20.258124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.329158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:20.336655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:20.336655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:20.336655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] PATH_INFO: `/v3/users/b1c98bf98b644312843a305486fb9f1a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:20.338270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c98bf98b644312843a305486fb9f1a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:20.340533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:20.345894 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1081/2162] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:20.353439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:20.356643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.442291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c98bf98b644312843a305486fb9f1a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.443370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c98bf98b644312843a305486fb9f1a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.459754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c98bf98b644312843a305486fb9f1a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.460764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c98bf98b644312843a305486fb9f1a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.473428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-49243b56-c75a-4db3-98b4-90bea6dadb53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c98bf98b644312843a305486fb9f1a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:20.476425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36f51e-b62f-4d3c-96d5-98742127affd None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:20.477110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:20.482877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:20.483987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:20.484717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:20.484984 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:20.488978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.522534 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1082/2163] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:20 2022] DELETE /identity/v3/users/b1c98bf98b644312843a305486fb9f1a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:20.527123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:20.527512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:20.527800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:20.543028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:20.546389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:20.552347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:20.553727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.566151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1082/2164] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:20.641122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1083/2165] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:20.651173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:20.653133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:20.674813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23034-8896-4a9f-b159-e3cd756f5147 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:20.759060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:20.789572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:20.790904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:20.790904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:20.790904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:20.813126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:20.824754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:20.839564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1083/2166] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:21.171986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:21.177198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:21.200337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffbd69-bfa8-45e7-bd96-ee7caabe5b22 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:21.206513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:21.208080 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:21.210243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:21.210489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] PATH_INFO: `/v3/projects/6bcaed2b380f44bc82c7537fa4aca464` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:21.211887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcaed2b380f44bc82c7537fa4aca464)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:21.214382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:21.286254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcaed2b380f44bc82c7537fa4aca464', 'request_id': 'req-4d050608-c7cf-41bb-b48d-75fba3e831fa'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.293321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcaed2b380f44bc82c7537fa4aca464', 'request_id': 'req-4d050608-c7cf-41bb-b48d-75fba3e831fa'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.300464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcaed2b380f44bc82c7537fa4aca464', 'request_id': 'req-4d050608-c7cf-41bb-b48d-75fba3e831fa'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.315707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] The token cache is being invalidate because project 6bcaed2b380f44bc82c7537fa4aca464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:21.316250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.317143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.319579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1084/2167] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:21 2022] DELETE /identity/v3/projects/6bcaed2b380f44bc82c7537fa4aca464 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:21.502484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:21.510110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:21.568810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd096d-7acb-441f-9690-d71561dd294d service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:21.694512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:21.695496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:21.695851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:21.696230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] PATH_INFO: `/v3/projects/163e2ec49f5d45d8a66c5c85d261a8e1` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:21.697653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] RBAC: Authorizing `identity:delete_project(project_id=163e2ec49f5d45d8a66c5c85d261a8e1)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:21.700867 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:21.792727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163e2ec49f5d45d8a66c5c85d261a8e1', 'request_id': 'req-e67bd8c9-9526-4598-8283-803b6e77395e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.803695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163e2ec49f5d45d8a66c5c85d261a8e1', 'request_id': 'req-e67bd8c9-9526-4598-8283-803b6e77395e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.810312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163e2ec49f5d45d8a66c5c85d261a8e1', 'request_id': 'req-e67bd8c9-9526-4598-8283-803b6e77395e'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.822447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] The token cache is being invalidate because project 163e2ec49f5d45d8a66c5c85d261a8e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:21.822739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.823397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.825000 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1084/2168] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:21 2022] DELETE /identity/v3/projects/163e2ec49f5d45d8a66c5c85d261a8e1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:21.829314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:21.830962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:21.844519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d050608-c7cf-41bb-b48d-75fba3e831fa admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:21.845888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:21.847614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:21.907707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:21.908603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:21.908915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:21.909188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] PATH_INFO: `/v3/users/347de8d8136a49a3bfae62fb5daa7809` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:21.910397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=347de8d8136a49a3bfae62fb5daa7809)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:21.912342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:21.914466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bd8c9-9526-4598-8283-803b6e77395e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:21.918377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:21.919038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:21.919445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:21.919924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:21.922276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:21.929874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:21.945960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:21.979323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347de8d8136a49a3bfae62fb5daa7809'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.980256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347de8d8136a49a3bfae62fb5daa7809'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:21.999690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347de8d8136a49a3bfae62fb5daa7809'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.000741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347de8d8136a49a3bfae62fb5daa7809'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.010823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347de8d8136a49a3bfae62fb5daa7809'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.025328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:22.026935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c47df9-0eae-4cd7-bb33-e9d488833ed7 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:22.030433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1085/2169] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:21 2022] DELETE /identity/v3/users/347de8d8136a49a3bfae62fb5daa7809 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:22.038698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1085/2170] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:22.399773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:22.402113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:22.499357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5f170-95f8-4917-9297-393c34708b58 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:22.504229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:22.505236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.505583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.505920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] PATH_INFO: `/v3/projects/928e8f9d41444147a1325fffc94858db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.507181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928e8f9d41444147a1325fffc94858db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:22.509341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:22.581588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928e8f9d41444147a1325fffc94858db', 'request_id': 'req-6a79a259-9523-4510-872e-45dbb3a1e428'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.588083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928e8f9d41444147a1325fffc94858db', 'request_id': 'req-6a79a259-9523-4510-872e-45dbb3a1e428'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.594226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928e8f9d41444147a1325fffc94858db', 'request_id': 'req-6a79a259-9523-4510-872e-45dbb3a1e428'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.606989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] The token cache is being invalidate because project 928e8f9d41444147a1325fffc94858db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:22.607271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.608193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6a79a259-9523-4510-872e-45dbb3a1e428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:22.610720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1086/2171] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:22 2022] DELETE /identity/v3/projects/928e8f9d41444147a1325fffc94858db => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:22.615679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.616195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.616195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.634498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:22.635728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:22.708361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1086/2172] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:22.716006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.716221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.716498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.734821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:22.736117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:22.744091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1087/2173] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:22.748321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:22.758670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66fb7-2f9e-4f1a-bd02-65588f5307b5 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:22.764216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:22.764912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.765196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.765472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.766214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:22.766521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:22.768255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:22.781442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1087/2174] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:22.785740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:22.795976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950831ff-64d3-46db-b6cd-f7e8d291f32e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:22.802174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:22.803725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.804006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.804273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.805009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:22.806988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:22.820940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:22.836835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1088/2175] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:22 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:22.840609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:22.850808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b675406-3b34-4755-a5c0-61e929f8e598 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:22.922605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:22.923418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.923699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.923977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.924705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:22.926664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:22.948477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1088/2176] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:22 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:22.952578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:22.963178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ce1f5-3922-45f5-ad21-20c1e9238abe admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:22.967892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:22.968515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:22.968824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:22.969096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:22.969978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:22.971696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:22.983086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1089/2177] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:22.987397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:22.998120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ce49-2832-43cf-b487-281aebbdb8fc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.003454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eef6f8-e52b-4f3f-a37d-0f572ccb883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.004174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eef6f8-e52b-4f3f-a37d-0f572ccb883a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.004515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eef6f8-e52b-4f3f-a37d-0f572ccb883a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.004801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eef6f8-e52b-4f3f-a37d-0f572ccb883a admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.005513 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eef6f8-e52b-4f3f-a37d-0f572ccb883a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.007136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eef6f8-e52b-4f3f-a37d-0f572ccb883a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.018225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1089/2178] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:23.022529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.033350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1b27-04ab-4c03-aa8c-5824c834ba4f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.038811 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.039706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.040001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.040256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] PATH_INFO: `/v3/projects/40bd48d766f64d089213cd82e4fea5e3/users/d4f673ba4d6142f5893c32b9e9e9bd3a/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.058157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] RBAC: Authorizing `identity:create_grant(project_id=40bd48d766f64d089213cd82e4fea5e3, user_id=d4f673ba4d6142f5893c32b9e9e9bd3a, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.060975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.074784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1090/2179] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:23 2022] PUT /identity/v3/projects/40bd48d766f64d089213cd82e4fea5e3/users/d4f673ba4d6142f5893c32b9e9e9bd3a/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:23.085605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.085893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.086193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.114935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] MFA Rules not processed for user `d4f673ba4d6142f5893c32b9e9e9bd3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:23.116228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:23.127546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:23.193864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1090/2180] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:23.198720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.266563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b55aec-9c02-4c7c-8fbe-5c72593022df admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.272366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.273075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.273382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.273634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.274333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.276264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.306993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1091/2181] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:23 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:23.311421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.324018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0090b-e611-495c-bbb9-a3575492e82d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.384979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.385655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.385939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.386216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.386915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.388899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.412277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1091/2182] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:23 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:23.416680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.428690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864dd15-5d14-48de-91a0-30c15804114c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.435090 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.435312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.435588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.435857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.436576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.438215 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.452891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1092/2183] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:23.454404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.464846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f794a0-93b7-496b-ae79-fe961e6c63c8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.469851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.470730 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.471083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.471363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] PATH_INFO: `/v3/projects/1741330a9c3f4590874781801d97af52/users/3951ca7b24e0439c927308707901ceba/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.487538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1741330a9c3f4590874781801d97af52, user_id=3951ca7b24e0439c927308707901ceba, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.490255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.504505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1092/2184] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:23 2022] PUT /identity/v3/projects/1741330a9c3f4590874781801d97af52/users/3951ca7b24e0439c927308707901ceba/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:23.508678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.519600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996c925-4f91-4dfc-9dda-19cdccdddfb1 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.585565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aca4a0-5db7-4eb2-be23-cc2abb1accfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.586238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca4a0-5db7-4eb2-be23-cc2abb1accfe admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.586514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca4a0-5db7-4eb2-be23-cc2abb1accfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.586780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca4a0-5db7-4eb2-be23-cc2abb1accfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.587516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aca4a0-5db7-4eb2-be23-cc2abb1accfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.589200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aca4a0-5db7-4eb2-be23-cc2abb1accfe admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.600223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1093/2185] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:23.605112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.617123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea18b70-d3dc-4ec8-9dd0-f34676b4acaf admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.624668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.624668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.624668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.624668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] PATH_INFO: `/v3/projects/1741330a9c3f4590874781801d97af52/users/3951ca7b24e0439c927308707901ceba/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.634046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1741330a9c3f4590874781801d97af52, user_id=3951ca7b24e0439c927308707901ceba, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.637008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.659335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1093/2186] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:23 2022] PUT /identity/v3/projects/1741330a9c3f4590874781801d97af52/users/3951ca7b24e0439c927308707901ceba/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:23.661318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.661781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.662169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.700355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] MFA Rules not processed for user `3951ca7b24e0439c927308707901ceba`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:23.701672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:23.742422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:23.811517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1094/2187] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:23.823020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:23.825363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:23.888493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0b1f7-4d88-43c4-9fc7-5ae10ee344d5 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:23.892295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f53db-402c-45ce-a1bb-502b69c1a204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:23.893102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f53db-402c-45ce-a1bb-502b69c1a204 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:23.893386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f53db-402c-45ce-a1bb-502b69c1a204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:23.893662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f53db-402c-45ce-a1bb-502b69c1a204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:23.902815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f53db-402c-45ce-a1bb-502b69c1a204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:23.904259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f53db-402c-45ce-a1bb-502b69c1a204 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:23.914615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1094/2188] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:23.943993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:24.009827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44126-0e10-4855-94ae-6b4239e5deff None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:24.015111 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:24.015987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:24.016261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:24.016603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] PATH_INFO: `/v3/users/d4f673ba4d6142f5893c32b9e9e9bd3a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:24.018027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f673ba4d6142f5893c32b9e9e9bd3a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:24.019975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:24.076167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f673ba4d6142f5893c32b9e9e9bd3a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.076990 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f673ba4d6142f5893c32b9e9e9bd3a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.090295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f673ba4d6142f5893c32b9e9e9bd3a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.091558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f673ba4d6142f5893c32b9e9e9bd3a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.101297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f673ba4d6142f5893c32b9e9e9bd3a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.114653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1095/2189] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:23 2022] DELETE /identity/v3/users/d4f673ba4d6142f5893c32b9e9e9bd3a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:24.119564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:24.120447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:24.121025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:24.136403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:24.142890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:24.144873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:24.214202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1095/2190] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:24.231854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:24.233412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:24.270171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:24.298528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad8e2-71e1-403a-a76b-cbf74e77f817 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:24.302334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:24.303027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:24.303312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:24.303584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:24.313355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:24.315151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:24.325387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1096/2191] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:24.628919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:24.630668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:24.651086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87e022-6c05-4e19-9433-ca7457430b6e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:24.657216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:24.658561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:24.659040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:24.659337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] PATH_INFO: `/v3/projects/40bd48d766f64d089213cd82e4fea5e3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:24.660728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=40bd48d766f64d089213cd82e4fea5e3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:24.662666 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:24.712298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40bd48d766f64d089213cd82e4fea5e3', 'request_id': 'req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.720260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40bd48d766f64d089213cd82e4fea5e3', 'request_id': 'req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.726479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40bd48d766f64d089213cd82e4fea5e3', 'request_id': 'req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.742271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] The token cache is being invalidate because project 40bd48d766f64d089213cd82e4fea5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:24.742564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.743220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.744986 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1096/2192] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:24 2022] DELETE /identity/v3/projects/40bd48d766f64d089213cd82e4fea5e3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:24.749116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:24.761231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55836f-31f8-4769-9f75-7afd295bb4c8 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:24.835851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:24.836802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:24.837083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:24.837461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] PATH_INFO: `/v3/users/3951ca7b24e0439c927308707901ceba` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:24.839092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3951ca7b24e0439c927308707901ceba)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:24.841878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:24.925594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3951ca7b24e0439c927308707901ceba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.927489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3951ca7b24e0439c927308707901ceba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.970173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3951ca7b24e0439c927308707901ceba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:24.971306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3951ca7b24e0439c927308707901ceba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.001445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1c269e9b-b9d7-42af-b923-df5b8ceedcaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3951ca7b24e0439c927308707901ceba'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.024676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1097/2193] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:24 2022] DELETE /identity/v3/users/3951ca7b24e0439c927308707901ceba => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:25.429860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:25.431905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:25.457254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36566b-a466-43da-bbc3-9bc9a8b7b0ce admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:25.514455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:25.515336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.515621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.515929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] PATH_INFO: `/v3/projects/1741330a9c3f4590874781801d97af52` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.517205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1741330a9c3f4590874781801d97af52)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:25.519136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:25.553886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:25.567038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1741330a9c3f4590874781801d97af52', 'request_id': 'req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.574529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1741330a9c3f4590874781801d97af52', 'request_id': 'req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.580892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1741330a9c3f4590874781801d97af52', 'request_id': 'req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.592295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] The token cache is being invalidate because project 1741330a9c3f4590874781801d97af52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:25.592588 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.593331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a40714b8-8bd3-450d-b414-bc1d9d58b1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:25.595020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1097/2194] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:25 2022] DELETE /identity/v3/projects/1741330a9c3f4590874781801d97af52 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:25.601501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.601806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.602187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.626746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:25.628115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:25.680515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:25.695000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1098/2195] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:25.703735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.704286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.704436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.728147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:25.729415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:25.740085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1098/2196] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:25.744659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:25.756014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702bba4-6c28-4b09-809b-5ae136127c4a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:25.761905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:25.762581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.763390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.763390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.763840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:25.764141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:25.765796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:25.782261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1099/2197] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:25.783835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:25.794048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b7a09-09eb-4f86-b974-4b2f84fcb27a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:25.800481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:25.801194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.801477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.801769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.802447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:25.804391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:25.836991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1099/2198] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:25 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:25.841664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:25.852650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3944f-a5bf-4889-8fb0-91047b3e32de admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:25.914000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:25.914694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.914976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.915251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.915950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:25.917941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:25.939209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1100/2199] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:25 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:25.943738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:25.954669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a547f4c-37f4-4384-bcd5-d7e42825744d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:25.960086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:25.962312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:25.962312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:25.962312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:25.962312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:25.964008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:25.979014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1100/2200] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:25.983743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:25.995467 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf206bd-2426-4951-b42f-c3e53bf691b4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:26.002358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4017d1-8f20-4800-b9ec-a3649daf121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:26.003042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4017d1-8f20-4800-b9ec-a3649daf121e admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:26.003410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4017d1-8f20-4800-b9ec-a3649daf121e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:26.003686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4017d1-8f20-4800-b9ec-a3649daf121e admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:26.004425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4017d1-8f20-4800-b9ec-a3649daf121e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:26.006104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4017d1-8f20-4800-b9ec-a3649daf121e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:26.021504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1101/2201] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:26.022767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:26.035803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2292e97-4199-4c7e-8994-b1ef5bda7bee admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:26.043501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443f9da-0231-4579-8f38-2fa3324d6476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:26.045129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443f9da-0231-4579-8f38-2fa3324d6476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:26.046081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443f9da-0231-4579-8f38-2fa3324d6476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:26.046506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443f9da-0231-4579-8f38-2fa3324d6476 admin admin] PATH_INFO: `/v3/projects/617b54d74adc4dc1add9ea3bad81282d/users/272fff49c5dc4db6a19d7bc6a6bf6bf6/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:26.074705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443f9da-0231-4579-8f38-2fa3324d6476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=617b54d74adc4dc1add9ea3bad81282d, user_id=272fff49c5dc4db6a19d7bc6a6bf6bf6, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:26.077758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443f9da-0231-4579-8f38-2fa3324d6476 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:26.095227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1101/2202] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:26 2022] PUT /identity/v3/projects/617b54d74adc4dc1add9ea3bad81282d/users/272fff49c5dc4db6a19d7bc6a6bf6bf6/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:26.106659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:26.106937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:26.107304 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:26.143653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] MFA Rules not processed for user `272fff49c5dc4db6a19d7bc6a6bf6bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:26.144926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:26.180074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:26.180365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:26.180744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:26.197921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:26.199143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:26.213268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1102/2203] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:26.225229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:26.227455 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:26.254409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1102/2204] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:26.285697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbc96-f693-4ff6-a930-3e4ac75e4857 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:26.289760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276484c5-4104-4e35-956e-47eea581bdfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:26.290131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276484c5-4104-4e35-956e-47eea581bdfd service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:26.290417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276484c5-4104-4e35-956e-47eea581bdfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:26.290693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276484c5-4104-4e35-956e-47eea581bdfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:26.296665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:26.299278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:26.300747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:26.301361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276484c5-4104-4e35-956e-47eea581bdfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:26.303506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276484c5-4104-4e35-956e-47eea581bdfd service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:26.314132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1103/2205] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:26.372863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:26.379023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b45912-4c70-4624-b279-e5fd6a4af18f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:26.386624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:26.387285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:26.387566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:26.387840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:26.396913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:26.398568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:26.407683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1103/2206] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:34:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 12:34:27.066794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.067301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.067409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.085475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:27.086729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:27.094177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1104/2207] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jun 30 12:34:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:27.107580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:27.118633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc94f-a0f8-4f93-9c47-49144554a900 service swift] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:27.134237 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44960f8-7f57-4ff7-aec2-c760d00bba37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:27.140970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44960f8-7f57-4ff7-aec2-c760d00bba37 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.141251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44960f8-7f57-4ff7-aec2-c760d00bba37 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.141634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44960f8-7f57-4ff7-aec2-c760d00bba37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.148283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:27.149738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:27.152009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44960f8-7f57-4ff7-aec2-c760d00bba37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:27.153718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44960f8-7f57-4ff7-aec2-c760d00bba37 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:27.165835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1104/2208] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:34:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:34:27.244920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678ea9-9334-4dcd-a400-910e3fb4eba3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:27.250504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:27.251879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.252267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.252696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] PATH_INFO: `/v3/users/03265311bba341488aa155a689ce56e7` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.254379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03265311bba341488aa155a689ce56e7)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:27.257049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:27.327222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03265311bba341488aa155a689ce56e7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:27.328221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03265311bba341488aa155a689ce56e7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:27.348515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03265311bba341488aa155a689ce56e7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:27.350139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03265311bba341488aa155a689ce56e7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:27.361749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-de1000f5-c03d-4f03-82c4-f1d73e271242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03265311bba341488aa155a689ce56e7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:27.385822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1105/2209] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:27 2022] DELETE /identity/v3/users/03265311bba341488aa155a689ce56e7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:27.390939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.391232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.391624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.411166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:27.412900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:27.442503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.443191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.443471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.462151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:27.463332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:27.485782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1105/2210] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:27.496275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:27.497771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:27.536226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1106/2211] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:34:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:27.551277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:27.553002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:27.563154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e517c-c5fb-4648-9c67-4ab88f9819e6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:27.567953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:27.572991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:27.573797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.574080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.574350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.587339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:27.589147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:27.600078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1106/2212] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:27.650660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d927c0-2b0c-464c-a99c-1a478b75eece None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:27.658679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:27.659366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:27.659980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:27.660232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:27.672224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:27.674844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:27.685694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1107/2213] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:34:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:34:27.999651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:28.001179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:28.034083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc183934-0e1b-4a6b-8d5e-a94dfa9e889c service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:28.071886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:28.073652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:28.073939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:28.074217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] PATH_INFO: `/v3/projects/f572e3eb5a0247f39dc2c225e52c0870` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:28.082971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f572e3eb5a0247f39dc2c225e52c0870)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:28.085064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:28.167212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f572e3eb5a0247f39dc2c225e52c0870', 'request_id': 'req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.184525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f572e3eb5a0247f39dc2c225e52c0870', 'request_id': 'req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.193529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f572e3eb5a0247f39dc2c225e52c0870', 'request_id': 'req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.229198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] The token cache is being invalidate because project f572e3eb5a0247f39dc2c225e52c0870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:28.233540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.234360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-a9c97e5d-e992-4f8c-9f01-90d2021bc3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.236299 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1107/2214] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:27 2022] DELETE /identity/v3/projects/f572e3eb5a0247f39dc2c225e52c0870 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:28.240154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:28.253178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e7904-d330-446b-9bd8-9557c1485063 service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:28.347981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:28.348269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:28.348774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:28.355994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:28.370528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:28.371733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:28.372718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:28.372845 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:28.372930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:28.373209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] PATH_INFO: `/v3/users/09afd78fd78c4d13b6aab3c43dba7613` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:28.374429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] RBAC: Authorizing `identity:delete_user(user_id=09afd78fd78c4d13b6aab3c43dba7613)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:28.376587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:28.439069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1108/2215] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jun 30 12:34:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:28.444443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09afd78fd78c4d13b6aab3c43dba7613'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.445374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09afd78fd78c4d13b6aab3c43dba7613'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.452852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:28.454332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:28.459143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09afd78fd78c4d13b6aab3c43dba7613'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.459822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09afd78fd78c4d13b6aab3c43dba7613'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.468346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09afd78fd78c4d13b6aab3c43dba7613'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:28.486580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1108/2216] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:28 2022] DELETE /identity/v3/users/09afd78fd78c4d13b6aab3c43dba7613 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:28.547909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777800-32bd-49ee-a1f6-3b2c1aa60360 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:28.645271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:28.645947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:28.646233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:28.646921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:28.648398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:28.730230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:28.734815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:28.738177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c1d50-99a1-4d28-91d8-1da7eb5fd517 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:28.748486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1109/2217] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:34:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 12:34:28.816235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:28.820517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:28.873995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:28.969267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2675f9-4872-42f2-838e-ceb8e2afa27a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:28.976858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:28.977950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:28.978481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:28.978870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] PATH_INFO: `/v3/projects/223bfff30b3c44c79261bc3aedcfd211` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:28.980562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=223bfff30b3c44c79261bc3aedcfd211)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:28.983122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.064652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223bfff30b3c44c79261bc3aedcfd211', 'request_id': 'req-996e8f00-2994-44e8-adbf-04fa04de3556'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:29.081604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223bfff30b3c44c79261bc3aedcfd211', 'request_id': 'req-996e8f00-2994-44e8-adbf-04fa04de3556'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:29.094274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223bfff30b3c44c79261bc3aedcfd211', 'request_id': 'req-996e8f00-2994-44e8-adbf-04fa04de3556'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:29.119707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] The token cache is being invalidate because project 223bfff30b3c44c79261bc3aedcfd211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:29.120043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:29.120894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-996e8f00-2994-44e8-adbf-04fa04de3556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:29.122999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1109/2218] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:28 2022] DELETE /identity/v3/projects/223bfff30b3c44c79261bc3aedcfd211 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:29.130452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b09771-9819-4680-9f71-831a3710686a None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.130740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b09771-9819-4680-9f71-831a3710686a None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.131014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b09771-9819-4680-9f71-831a3710686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.153576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-45b09771-9819-4680-9f71-831a3710686a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:29.154564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-45b09771-9819-4680-9f71-831a3710686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:29.275877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1110/2219] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:29.284594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.285123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.285227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.310323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:29.311574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:29.325625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1110/2220] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:29.330850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b09771-9819-4680-9f71-831a3710686a None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.351260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b09771-9819-4680-9f71-831a3710686a None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:29.356306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:29.356531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.356833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.357111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.357842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:29.358145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:29.359747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.384447 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1111/2221] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:34:29.392350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.405441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8998e-caf5-44ce-a82e-eed24e35ede5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:29.411288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:29.412300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.412854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.413288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.414129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:29.416209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.453528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1111/2222] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:29 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:29.454035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.465230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2d48-ba66-42a5-88a0-352938f94b85 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:29.585373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:29.586537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.586889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.587223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.588135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:29.590375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.614033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1112/2223] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:29 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:29.618684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.631177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9b667-d6cf-4f40-87c3-bec4c696c004 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:29.635735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:29.636512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.636869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.637139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.637896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:29.639564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.652774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1112/2224] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:29.657169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.669538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803db59-4ff6-47f5-a3a4-0fa4f22686c5 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:29.675744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc020149-4de7-42e3-ad1e-5c5fcd5ab75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:29.676537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020149-4de7-42e3-ad1e-5c5fcd5ab75f admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.676912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020149-4de7-42e3-ad1e-5c5fcd5ab75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.677245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020149-4de7-42e3-ad1e-5c5fcd5ab75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.678158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020149-4de7-42e3-ad1e-5c5fcd5ab75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:29.680006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020149-4de7-42e3-ad1e-5c5fcd5ab75f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.694535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1113/2225] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:29.704922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.713631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d77a5-66ec-4ffb-8803-140d2230c782 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:29.720133 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:29.721325 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.721675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.722070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] PATH_INFO: `/v3/projects/2b88c87122614037ae0fb1ee5ee9d19d/users/5609e97af7c046cab470e06b6d756e83/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.738907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b88c87122614037ae0fb1ee5ee9d19d, user_id=5609e97af7c046cab470e06b6d756e83, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:29.742132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:29.760249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1113/2226] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:29 2022] PUT /identity/v3/projects/2b88c87122614037ae0fb1ee5ee9d19d/users/5609e97af7c046cab470e06b6d756e83/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:29.773482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:29.774011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:29.774309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:29.807893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] MFA Rules not processed for user `5609e97af7c046cab470e06b6d756e83`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:29.809574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:29.901944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:29.932921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1114/2227] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:29.946490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:29.954891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:30.049086 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:30.059007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eae28-b9e4-450d-8496-59b595013608 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:30.065659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20934aa8-6138-496b-a9f1-942aa3009ce7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:30.066451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20934aa8-6138-496b-a9f1-942aa3009ce7 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:30.066837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20934aa8-6138-496b-a9f1-942aa3009ce7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:30.067250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20934aa8-6138-496b-a9f1-942aa3009ce7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:30.088112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20934aa8-6138-496b-a9f1-942aa3009ce7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:30.089936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20934aa8-6138-496b-a9f1-942aa3009ce7 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:30.101884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1114/2228] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:30.123244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:30.217998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd3fe-9c3a-433d-a914-14dd7fda4ff2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:30.224816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:30.226241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:30.226790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:30.227131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] PATH_INFO: `/v3/users/5609e97af7c046cab470e06b6d756e83` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:30.228876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5609e97af7c046cab470e06b6d756e83)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:30.231269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:30.294798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5609e97af7c046cab470e06b6d756e83'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:30.296158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5609e97af7c046cab470e06b6d756e83'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:30.312665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5609e97af7c046cab470e06b6d756e83'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:30.313542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5609e97af7c046cab470e06b6d756e83'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:30.323708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5609e97af7c046cab470e06b6d756e83'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:30.340491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1115/2229] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:30 2022] DELETE /identity/v3/users/5609e97af7c046cab470e06b6d756e83 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:30.345416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:30.345964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:30.346157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:30.364091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:30.365397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:30.468548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1115/2230] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:30.487941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:30.490190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:30.574001 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334081b4-605d-4616-aa19-143b7c654068 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:30.578042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645b629-edac-48a0-a54b-85f67d0381ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:30.578553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b629-edac-48a0-a54b-85f67d0381ee service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:30.578861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b629-edac-48a0-a54b-85f67d0381ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:30.579148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b629-edac-48a0-a54b-85f67d0381ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:30.589608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645b629-edac-48a0-a54b-85f67d0381ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:30.591304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645b629-edac-48a0-a54b-85f67d0381ee service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:30.603648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1116/2231] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:30.944894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:30.946830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:30.973610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b620d-5913-4b94-8604-f6384a9ab691 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:30.979727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:30.980758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:30.981830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:30.982189 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] PATH_INFO: `/v3/projects/2b88c87122614037ae0fb1ee5ee9d19d` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:30.984543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b88c87122614037ae0fb1ee5ee9d19d)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:30.987323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.056338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b88c87122614037ae0fb1ee5ee9d19d', 'request_id': 'req-3fe60c77-da46-4195-b984-5a72be91cee5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.063221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b88c87122614037ae0fb1ee5ee9d19d', 'request_id': 'req-3fe60c77-da46-4195-b984-5a72be91cee5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.069798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b88c87122614037ae0fb1ee5ee9d19d', 'request_id': 'req-3fe60c77-da46-4195-b984-5a72be91cee5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.084751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] The token cache is being invalidate because project 2b88c87122614037ae0fb1ee5ee9d19d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:31.085046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.085713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3fe60c77-da46-4195-b984-5a72be91cee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.091063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1116/2232] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:30 2022] DELETE /identity/v3/projects/2b88c87122614037ae0fb1ee5ee9d19d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:31.093628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768a1be-e9f7-4648-880b-c80583256c4e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.093990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768a1be-e9f7-4648-880b-c80583256c4e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.095066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768a1be-e9f7-4648-880b-c80583256c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.116286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-4768a1be-e9f7-4648-880b-c80583256c4e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:31.117946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4768a1be-e9f7-4648-880b-c80583256c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.125854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:31.208872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1117/2233] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:31.215619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.215979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.216329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.264957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:31.266246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.275780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1117/2234] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:31.277327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ab6dc-4d01-486e-bc03-b34290e59505 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.277620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ab6dc-4d01-486e-bc03-b34290e59505 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.277900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ab6dc-4d01-486e-bc03-b34290e59505 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.280497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1118/2235] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:34:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:34:31.281432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.292740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc5565-4bee-46c3-8eee-9eee1848a41b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.304408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.304932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.305025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.309832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.310888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.311329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.311717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.312488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:31.312800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.315480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.335968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1118/2236] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:31.340151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.341785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] MFA Rules not processed for user `0783ae884dd1434bbfc76b91457858f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:31.343025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.351360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97662a3-c16c-452d-9f40-b84ec7492053 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.356407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.357198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.357499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.357771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.358476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.360500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.404333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1119/2237] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:31 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:31.405816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.446716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1119/2238] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:31.448067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.451534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.456224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:31.466634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b501646-b0b5-4adf-bc6c-8d825657fd96 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.483621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b7f6a-7aac-4382-bf8a-c0b38ca4b709 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.489703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.490523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.490852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.491147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.491924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.494026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.517265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1120/2239] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:31 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:31.518106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.519695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.534869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.536045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.536365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.536878 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] PATH_INFO: `/v3/users/3b688425466947198520ec7554bb49f7` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.538354 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b688425466947198520ec7554bb49f7)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.540531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.601088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b688425466947198520ec7554bb49f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.601749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b688425466947198520ec7554bb49f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.604147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:31.605112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143f13e-1382-489a-a4b8-6424741a096a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.611358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.611835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.612115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.612387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.614248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.617744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b688425466947198520ec7554bb49f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.620101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b688425466947198520ec7554bb49f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.628877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b688425466947198520ec7554bb49f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.630365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.632536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.643570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1121/2241] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:31.643741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1120/2241] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:31 2022] DELETE /identity/v3/users/3b688425466947198520ec7554bb49f7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:31.644390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.645158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.645760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.646856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.668852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe96591-dd10-45dd-849e-d7586548fa9a service placement] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.734295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22246-efe0-4e54-b3b5-4ab19f3fef1f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.744456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.745506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.745816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.746863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.747553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.747826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.748094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.748869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.749077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] PATH_INFO: `/v3/users/c969fad4feb7468abd62714899d3fc2b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.750512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.750969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c969fad4feb7468abd62714899d3fc2b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.753138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.763784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1122/2242] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:31.767666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.777858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0733d5-b4d6-4112-8708-9e9e8d05f428 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.783695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.784434 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.784797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.785077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.785846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.787500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.800362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1123/2243] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:31.804436 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.810027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c969fad4feb7468abd62714899d3fc2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.810905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c969fad4feb7468abd62714899d3fc2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.815777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5754f7-5346-43aa-b0a6-7d77ccb7e080 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.822099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:31.822881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.823162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.823586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] PATH_INFO: `/v3/projects/f228bf334169493c955db880685bb49d/users/115310c81bcb45a1b23cfef52cbdad10/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.848324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c969fad4feb7468abd62714899d3fc2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.849127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c969fad4feb7468abd62714899d3fc2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.853562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f228bf334169493c955db880685bb49d, user_id=115310c81bcb45a1b23cfef52cbdad10, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:31.856929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:31.863849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-7861ca9c-0e7f-4ed0-ac74-225ed36c995e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c969fad4feb7468abd62714899d3fc2b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:31.876619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1124/2244] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:31 2022] PUT /identity/v3/projects/f228bf334169493c955db880685bb49d/users/115310c81bcb45a1b23cfef52cbdad10/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:31.885496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:31.885496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.893637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1121/2245] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:31 2022] DELETE /identity/v3/users/c969fad4feb7468abd62714899d3fc2b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:31.896682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:31.897794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaa2bd-6bae-44a6-b1d5-7904c46116bf None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:31.898098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaa2bd-6bae-44a6-b1d5-7904c46116bf None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:31.898389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaa2bd-6bae-44a6-b1d5-7904c46116bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:31.932015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f0eaa2bd-6bae-44a6-b1d5-7904c46116bf None None] MFA Rules not processed for user `115310c81bcb45a1b23cfef52cbdad10`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:31.933265 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f0eaa2bd-6bae-44a6-b1d5-7904c46116bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:31.953966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1b398-55cf-47d7-a203-2e8487c75a0b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:31.966352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:32.000961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1122/2246] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:32.002279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.002566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.002837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.020262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:32.021203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.021517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.022075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.022410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.022704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] PATH_INFO: `/v3/projects/8fe4f134e4084a878a5e5cf59d96630f` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.023918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe4f134e4084a878a5e5cf59d96630f)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.026158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.034539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1123/2247] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:32.037565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.039865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.080105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe4f134e4084a878a5e5cf59d96630f', 'request_id': 'req-0ebeabdc-311d-4837-9d12-a29be7a889b6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.086382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe4f134e4084a878a5e5cf59d96630f', 'request_id': 'req-0ebeabdc-311d-4837-9d12-a29be7a889b6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.092467 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe4f134e4084a878a5e5cf59d96630f', 'request_id': 'req-0ebeabdc-311d-4837-9d12-a29be7a889b6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.105087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd0ab0-59fc-4faa-8405-c1d044739625 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.106806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] The token cache is being invalidate because project 8fe4f134e4084a878a5e5cf59d96630f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:32.107099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.107744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.109405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1125/2248] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:31 2022] DELETE /identity/v3/projects/8fe4f134e4084a878a5e5cf59d96630f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:32.110358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.111668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.159582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.160239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.160520 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.160832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.163379 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.168591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:32.179996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeabdc-311d-4837-9d12-a29be7a889b6 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.186038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.186310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.186494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.186774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.190668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.236100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.237353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e111d01-5071-4b3f-a685-78b8ba0d6985 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.247129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1124/2249] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:32.248391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979343af-053e-4481-bf15-9112d3e4839e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.248885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979343af-053e-4481-bf15-9112d3e4839e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.248994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979343af-053e-4481-bf15-9112d3e4839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.268438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.270098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7b253-1282-4642-b43a-1600fa879fa8 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.272497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-979343af-053e-4481-bf15-9112d3e4839e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:32.274133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-979343af-053e-4481-bf15-9112d3e4839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.283126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1126/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:32.287414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1125/2251] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:32.296855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.297150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.297432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.326445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:32.328978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.341299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1127/2252] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:32.346434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979343af-053e-4481-bf15-9112d3e4839e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.362373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979343af-053e-4481-bf15-9112d3e4839e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.369290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.370550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.371027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.372160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.373094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:32.373554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.375936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.394328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1126/2253] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:32.398647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.412311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1ba9a-0017-4137-b718-f90804ee6e7c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.421036 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.421740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.422068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.422358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.423253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.427673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.468277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1128/2254] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:32.473200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.490049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121a9e9-b7bc-48fa-87f1-75afbe3b7d02 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.526334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:32.574028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.576564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.585640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.586850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.587698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.587698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.599294 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.601809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.643877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1127/2255] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:32.649002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.667412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e850c9d-0cd7-43c1-99ab-d8adb24713b6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.678926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d556fa-17e6-4b80-8774-abe7ea688c26 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.679909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.680611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.683032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.683443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.684839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.685737 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.685856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.686386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.686752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] PATH_INFO: `/v3/projects/48748543262c417d9089326f428e6b63` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.688226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48748543262c417d9089326f428e6b63)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.689150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.690469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.712221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1128/2256] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:32.717329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.749993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac996787-7ef5-42bc-bd65-e4c1b2c7e707 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.762611 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.763358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.763649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.764241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.765044 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.768072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.778677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48748543262c417d9089326f428e6b63', 'request_id': 'req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.787075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1129/2257] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:32.791854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48748543262c417d9089326f428e6b63', 'request_id': 'req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.792677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:32.799878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48748543262c417d9089326f428e6b63', 'request_id': 'req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.814507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4956-4ba1-46fc-b883-776e4729443a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:32.815323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:32.822280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] The token cache is being invalidate because project 48748543262c417d9089326f428e6b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:32.822665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.823440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2ca3a8ad-fb4e-4923-a3bf-3880fda43a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:32.825179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1129/2258] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:32 2022] DELETE /identity/v3/projects/48748543262c417d9089326f428e6b63 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:32.832137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c6b7f-f527-4bcf-b01e-74af79981248 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.833287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c6b7f-f527-4bcf-b01e-74af79981248 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.833612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c6b7f-f527-4bcf-b01e-74af79981248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.855588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-974c6b7f-f527-4bcf-b01e-74af79981248 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:32.857233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-974c6b7f-f527-4bcf-b01e-74af79981248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.888304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed09817-7cdc-44c2-95ba-8b9454c9740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:32.889679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed09817-7cdc-44c2-95ba-8b9454c9740e admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.889978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed09817-7cdc-44c2-95ba-8b9454c9740e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.890256 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed09817-7cdc-44c2-95ba-8b9454c9740e admin admin] PATH_INFO: `/v3/projects/63fc82464e41424f93ceda870cd0c1dd/users/c468edd5263442e4825dcc4711838579/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.905613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed09817-7cdc-44c2-95ba-8b9454c9740e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fc82464e41424f93ceda870cd0c1dd, user_id=c468edd5263442e4825dcc4711838579, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:32.908560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed09817-7cdc-44c2-95ba-8b9454c9740e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:32.920154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1130/2259] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:32.923342 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1130/2260] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:32 2022] PUT /identity/v3/projects/63fc82464e41424f93ceda870cd0c1dd/users/c468edd5263442e4825dcc4711838579/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:32.928258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.928553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.928867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.933280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:32.933568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:32.933844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:32.950035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:32.951282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:32.959856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] MFA Rules not processed for user `c468edd5263442e4825dcc4711838579`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:32.961236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:33.016991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1131/2261] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:33.018209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.022849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:33.027037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1131/2262] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:33.027919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8c97e-7db6-45f3-bf59-94176d7cd1f5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.033541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.034326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.034622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.034984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.035866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:33.036072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.038055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.041637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.043969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:33.052249 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1132/2263] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:33.056730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.067260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbdae6-17d1-4f76-9d5c-d4f8ca14f17f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.072300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.073008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.073297 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.073595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.074309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.076243 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.105975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1133/2264] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:33 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:33.109831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.130976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963610-ac1f-4816-b5f4-a3f1ea27a483 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.172694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb47e1-db07-4693-a765-5e8c66fcc5d2 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.177880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.178725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.179101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.179477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.180594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.182435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.192444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:33.194772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47d5cc-e407-4eda-adb9-0f22f84b7e77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.195718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47d5cc-e407-4eda-adb9-0f22f84b7e77 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.196199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47d5cc-e407-4eda-adb9-0f22f84b7e77 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.196741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47d5cc-e407-4eda-adb9-0f22f84b7e77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.205174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1134/2265] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:33 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:33.209305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.219329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f72a46-b839-49f1-a97c-40c5152d05bd admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.224490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.225192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.225474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.225747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.226472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.228075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.238132 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1135/2266] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:33.242026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.251460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24c1eb-9ff4-4987-8c47-e7ca68b6b8f9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.256510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.257198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.257489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.257760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.258473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.259920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47d5cc-e407-4eda-adb9-0f22f84b7e77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.260057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.261542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47d5cc-e407-4eda-adb9-0f22f84b7e77 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.270528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1136/2267] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:33.270939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1132/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:33.274492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.284404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf002-a81b-4049-b2ed-9b547697a2e4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.289178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.290074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.290353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.290643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] PATH_INFO: `/v3/projects/4a42d5073b0f4e4c857d794aa8d30393/users/1fc65fea19f94fd18d83ad8ce3bbb476/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.305813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a42d5073b0f4e4c857d794aa8d30393, user_id=1fc65fea19f94fd18d83ad8ce3bbb476, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.308497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.323708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1137/2269] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:33 2022] PUT /identity/v3/projects/4a42d5073b0f4e4c857d794aa8d30393/users/1fc65fea19f94fd18d83ad8ce3bbb476/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:33.329746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f9a4-4120-4d28-b454-8369e076917b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.330127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f9a4-4120-4d28-b454-8369e076917b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.330518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261f9a4-4120-4d28-b454-8369e076917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.350219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0261f9a4-4120-4d28-b454-8369e076917b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:33.351428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0261f9a4-4120-4d28-b454-8369e076917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:33.384775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:33.415721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1133/2270] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:33.426854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:33.429449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:33.493081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087f9b-93c0-49ee-b80d-938a5d8afda0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:33.498386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:33.499108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:33.499412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:33.499707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:33.510700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:33.512264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:33.524327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1138/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:37.994617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:37.995127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:37.995308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.024774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] MFA Rules not processed for user `1fc65fea19f94fd18d83ad8ce3bbb476`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:38.026042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:38.123704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1134/2272] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:38.128363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.138318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0d1bc-1022-4cbd-b16b-ac1cc45da364 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.143157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.143935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.144280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.144618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.145375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.147354 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.182227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1139/2273] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:38 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:38.186350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.197783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490fa70-14b2-40af-b819-1aa3e0695022 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.263754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.264448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.264765 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.265043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.265745 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.267729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.289579 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1135/2274] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:38 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:38.295512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.307856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adef34-173d-4c3f-b514-e94403d1afe3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.313007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.313816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.314164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.314448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.315188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.316836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.332433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1140/2275] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:38.333707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.346111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cd33e-f450-4b8b-a105-aa0275e9e561 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.347518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.349466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:38.353011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.353929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.354208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.354483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] PATH_INFO: `/v3/projects/52a21e84f1ee4a9c847118c7570d23b4/users/830320d05a1b426bac4849fb2f541f8c/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.363153 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87db3-ec4e-46dc-b8ef-e8c0f1c44d50 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.368362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.369311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.369614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.369888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] PATH_INFO: `/v3/users/115310c81bcb45a1b23cfef52cbdad10` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.371112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=115310c81bcb45a1b23cfef52cbdad10)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.372877 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a21e84f1ee4a9c847118c7570d23b4, user_id=830320d05a1b426bac4849fb2f541f8c, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.373088 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.375628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.391391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1136/2276] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:38 2022] PUT /identity/v3/projects/52a21e84f1ee4a9c847118c7570d23b4/users/830320d05a1b426bac4849fb2f541f8c/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:38.395071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.433392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:38.436715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115310c81bcb45a1b23cfef52cbdad10'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:38.437604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115310c81bcb45a1b23cfef52cbdad10'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:38.454171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115310c81bcb45a1b23cfef52cbdad10'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:38.454922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115310c81bcb45a1b23cfef52cbdad10'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:38.464706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b208edb8-eefd-493a-ad16-3dab799505e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115310c81bcb45a1b23cfef52cbdad10'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:38.466893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:38.472837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e433-9cc8-4c3e-8ffc-1ad8d94c68f7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.481538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.482231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.482511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.485193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.485941 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.487548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.490420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1141/2277] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:38 2022] DELETE /identity/v3/users/115310c81bcb45a1b23cfef52cbdad10 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:38.495442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.495734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.496151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.530089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1137/2278] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:38.530661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:38.532094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:38.534529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.536106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:38.553031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5674-f558-4578-a750-4b76a133ad3e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.619467 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1142/2279] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:38.629102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:38.630591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:38.634154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.635203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.635486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.635762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] PATH_INFO: `/v3/projects/52a21e84f1ee4a9c847118c7570d23b4/users/830320d05a1b426bac4849fb2f541f8c/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.638164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a21e84f1ee4a9c847118c7570d23b4, user_id=830320d05a1b426bac4849fb2f541f8c, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.640897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.657297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1138/2280] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:38 2022] PUT /identity/v3/projects/52a21e84f1ee4a9c847118c7570d23b4/users/830320d05a1b426bac4849fb2f541f8c/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:38.698100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47367ef6-cadc-419f-bd9d-3134565890ff None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:38.758032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b411790-17e1-4810-a811-ed015974efc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:38.758706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b411790-17e1-4810-a811-ed015974efc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:38.759015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b411790-17e1-4810-a811-ed015974efc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:38.759284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b411790-17e1-4810-a811-ed015974efc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:38.859826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b411790-17e1-4810-a811-ed015974efc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:38.861707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b411790-17e1-4810-a811-ed015974efc8 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:38.866229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:38.907226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1143/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:39.601350 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:39.603415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:39.625270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d943d91-604a-4194-a0d0-25680ffed547 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:39.637517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:39.638416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:39.638696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:39.638967 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] PATH_INFO: `/v3/projects/f228bf334169493c955db880685bb49d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:39.640175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f228bf334169493c955db880685bb49d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:39.642172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:39.708235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f228bf334169493c955db880685bb49d', 'request_id': 'req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:39.730351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f228bf334169493c955db880685bb49d', 'request_id': 'req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:39.742051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f228bf334169493c955db880685bb49d', 'request_id': 'req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:39.768769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] The token cache is being invalidate because project f228bf334169493c955db880685bb49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:39.769458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:39.770436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d7df5cd5-870e-4eb0-a8cf-9207835fb02c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:39.772077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1139/2282] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:39 2022] DELETE /identity/v3/projects/f228bf334169493c955db880685bb49d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:39.777130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:39.777591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:39.777672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:39.801976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:39.803340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:39.877843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1144/2283] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:39.886993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:39.887262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:39.887855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:39.911906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:39.913374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:39.922594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1140/2284] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:39.926995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:39.938931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb01db-0002-4f4d-88d0-534fb46eec12 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:39.944074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:39.944931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:39.945375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:39.945667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:39.946517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:39.946986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:39.948777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:39.970412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1145/2285] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:39.972659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:39.987264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c500178-fac3-4a55-b5de-ea4d1ccc327e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:39.993455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:39.995020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:39.996048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:39.996493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:39.997276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:39.999243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:40.040683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1141/2286] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:39 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:40.045991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:40.057381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da521bcd-70c4-44c5-8d9e-6a23f4495346 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:40.137902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:40.139155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:40.139458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:40.139852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:40.140813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:40.143312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:40.171863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1146/2287] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:40 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:40.176293 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:40.191964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a67f-a045-4156-86f9-1e2f7ab0dcb4 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:40.203890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:40.204826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:40.205259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:40.205708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:40.207297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:40.209008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:40.224239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1142/2288] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:40.228475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:40.253748 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f328d-e5a3-48b0-a26d-541b3a79ff01 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:40.281824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd9610-823c-4ecc-b906-89c657f3bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:40.282295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd9610-823c-4ecc-b906-89c657f3bb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:40.282775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd9610-823c-4ecc-b906-89c657f3bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:40.283056 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd9610-823c-4ecc-b906-89c657f3bb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:40.283819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd9610-823c-4ecc-b906-89c657f3bb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:40.285500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd9610-823c-4ecc-b906-89c657f3bb2b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:40.307407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1147/2289] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:40.311413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:40.319853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:40.329911 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd0212-be00-43e2-92ce-29e10fcf60b3 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:40.346709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:40.347920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:40.348374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:40.348821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] PATH_INFO: `/v3/projects/a338f94294fe4a00b2921c62fdb0ad85/users/01fa74b7c1ee418fa9938f1f9f6bebc4/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:40.384750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a338f94294fe4a00b2921c62fdb0ad85, user_id=01fa74b7c1ee418fa9938f1f9f6bebc4, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:40.394270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:40.435363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1143/2290] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:40 2022] PUT /identity/v3/projects/a338f94294fe4a00b2921c62fdb0ad85/users/01fa74b7c1ee418fa9938f1f9f6bebc4/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:40.435629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57815c3b-268d-4669-b99c-041d8ed35a73 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:40.435943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57815c3b-268d-4669-b99c-041d8ed35a73 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:40.436245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57815c3b-268d-4669-b99c-041d8ed35a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:40.475827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-57815c3b-268d-4669-b99c-041d8ed35a73 None None] MFA Rules not processed for user `01fa74b7c1ee418fa9938f1f9f6bebc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:40.478021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-57815c3b-268d-4669-b99c-041d8ed35a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:40.578457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:40.596481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1148/2291] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:40.608180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:40.611359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:40.710298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fe5d6-034e-4f5f-b29c-4699e877644c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:40.723719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:40.724408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:40.724741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:40.725003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:40.745084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:40.748060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:40.762750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1144/2292] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:44.673322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7cbe7-b37a-4873-9315-b2362f7e95fc None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:44.674207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7cbe7-b37a-4873-9315-b2362f7e95fc None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:44.674491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7cbe7-b37a-4873-9315-b2362f7e95fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:44.703682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-dcf7cbe7-b37a-4873-9315-b2362f7e95fc None None] MFA Rules not processed for user `830320d05a1b426bac4849fb2f541f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:44.704937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-dcf7cbe7-b37a-4873-9315-b2362f7e95fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:44.773671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1149/2293] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:44.786194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:44.787593 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:44.882403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5230a8-9218-4bd2-8067-90dbe620cc62 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:44.888225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:44.889434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:44.889434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:44.889630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:44.892589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:44.966483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:44.967936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:44.977785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1145/2294] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:45.124004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:45.124352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:45.124558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:45.148441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:45.149870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:45.218015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1150/2295] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:34:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:45.231261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:45.232846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:45.278703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:45.309206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881bb7a-0c8d-460f-9b3f-3def6729638e service glance] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:45.314468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d124cb-cc17-43ff-b2c7-53827d02825e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:45.315131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d124cb-cc17-43ff-b2c7-53827d02825e service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:45.315392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d124cb-cc17-43ff-b2c7-53827d02825e service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:45.315662 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d124cb-cc17-43ff-b2c7-53827d02825e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:45.324781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d124cb-cc17-43ff-b2c7-53827d02825e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:45.326390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d124cb-cc17-43ff-b2c7-53827d02825e service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:45.335478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1146/2296] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:34:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:34:52.667511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:52.669177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:52.763823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f487a-8dfc-4987-b2ef-c514b545111f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:52.771314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:52.772556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:52.773141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:52.773595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] PATH_INFO: `/v3/users/67fafa178edd4acf8da8d67bb6b24761` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:52.775185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] RBAC: Authorizing `identity:delete_user(user_id=67fafa178edd4acf8da8d67bb6b24761)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:52.777757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:52.831117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:52.864864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67fafa178edd4acf8da8d67bb6b24761'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:52.865533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67fafa178edd4acf8da8d67bb6b24761'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:52.883679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67fafa178edd4acf8da8d67bb6b24761'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:52.884427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67fafa178edd4acf8da8d67bb6b24761'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:52.913001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67fafa178edd4acf8da8d67bb6b24761'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:52.932176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1151/2297] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:52 2022] DELETE /identity/v3/users/67fafa178edd4acf8da8d67bb6b24761 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:52.936478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:52.937446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:52.937876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:52.959753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:52.961056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:53.036433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1147/2298] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:53.050736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:53.052155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:53.135840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c430e-4129-40f4-98ff-24bd10be566e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:53.137874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85163a6e-669b-487b-af1f-67bf70c27cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:53.138543 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85163a6e-669b-487b-af1f-67bf70c27cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:53.138831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85163a6e-669b-487b-af1f-67bf70c27cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:53.139279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85163a6e-669b-487b-af1f-67bf70c27cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:53.154228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85163a6e-669b-487b-af1f-67bf70c27cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:53.155693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85163a6e-669b-487b-af1f-67bf70c27cc5 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:53.190108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1152/2299] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:53.251474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:53.252108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:53.277259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbe2da-49f0-4582-9bbe-4de3a6b8c772 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:53.283861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:53.285162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:53.286205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:53.286602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] PATH_INFO: `/v3/users/c468edd5263442e4825dcc4711838579` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:53.288521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c468edd5263442e4825dcc4711838579)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:53.290671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:53.350922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c468edd5263442e4825dcc4711838579'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.352085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c468edd5263442e4825dcc4711838579'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.368871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c468edd5263442e4825dcc4711838579'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.370007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c468edd5263442e4825dcc4711838579'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.385748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c468edd5263442e4825dcc4711838579'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.409157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:53.409448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:53.409726 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:53.411692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1148/2300] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:53 2022] DELETE /identity/v3/users/c468edd5263442e4825dcc4711838579 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:53.451553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:53.453206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:53.571922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1153/2301] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:53.581511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:53.584747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:53.631601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:53.633697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:53.656238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaef1a2-cc6c-455b-983c-59293a54453f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:53.675283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:53.676225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:53.677945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:53.678260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:53.678546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] PATH_INFO: `/v3/projects/79f02ad55da84c41834fd71c04d576ab` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:53.681581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f02ad55da84c41834fd71c04d576ab)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:53.689849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:53.701834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7ae91-ae5e-4547-bf66-e585691f5df5 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:53.715034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d56c-1c42-4fc0-9c20-d77c5416b29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:53.716033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d56c-1c42-4fc0-9c20-d77c5416b29f service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:53.716615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d56c-1c42-4fc0-9c20-d77c5416b29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:53.717128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d56c-1c42-4fc0-9c20-d77c5416b29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:53.731041 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d56c-1c42-4fc0-9c20-d77c5416b29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:53.732521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d56c-1c42-4fc0-9c20-d77c5416b29f service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:53.769870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1149/2302] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:53.795063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:53.810113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f02ad55da84c41834fd71c04d576ab', 'request_id': 'req-eb1c2267-7ade-4455-83d2-c2a783798d8b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.824609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f02ad55da84c41834fd71c04d576ab', 'request_id': 'req-eb1c2267-7ade-4455-83d2-c2a783798d8b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.836616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f02ad55da84c41834fd71c04d576ab', 'request_id': 'req-eb1c2267-7ade-4455-83d2-c2a783798d8b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.857792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] The token cache is being invalidate because project 79f02ad55da84c41834fd71c04d576ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:53.858241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.858967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-eb1c2267-7ade-4455-83d2-c2a783798d8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:53.871145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:53.871145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:53.871145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:53.871382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1154/2303] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:53 2022] DELETE /identity/v3/projects/79f02ad55da84c41834fd71c04d576ab => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:53.892445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:53.894902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.017023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1150/2304] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:54.024869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.025416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.025843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.055163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:54.057024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.069059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1155/2305] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:54.072919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.085566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05f013-f18c-4c9e-aba0-e398ef111bd6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.094591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.095429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.095840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.096729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.097846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:54.098298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.100145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.121531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1151/2306] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:54.122411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.122962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.124563 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.134980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988f864-df79-45b6-9f20-eed63c3a5f81 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.140734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.141670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.142045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.142543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.143898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.145468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.163606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b3509-7f2e-4c80-9acf-91cbeb8deea4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.171178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.175398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.175689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.175969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] PATH_INFO: `/v3/projects/63fc82464e41424f93ceda870cd0c1dd` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.180074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fc82464e41424f93ceda870cd0c1dd)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.182034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.197242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1152/2307] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:54.201482 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.245783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fc82464e41424f93ceda870cd0c1dd', 'request_id': 'req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:54.256162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fc82464e41424f93ceda870cd0c1dd', 'request_id': 'req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:54.266103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fc82464e41424f93ceda870cd0c1dd', 'request_id': 'req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:54.286382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] The token cache is being invalidate because project 63fc82464e41424f93ceda870cd0c1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:34:54.286663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:54.289815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e7d3a1fc-4c8d-4888-a8d7-a0e4fb1a94db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:54.296476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1156/2308] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:34:54 2022] DELETE /identity/v3/projects/63fc82464e41424f93ceda870cd0c1dd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:54.299315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241a830-7681-451d-b1b5-6cbd04de3e5a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.301643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ef69d-953b-4f01-9481-6367f1f8c47d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.302035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ef69d-953b-4f01-9481-6367f1f8c47d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.302451 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ef69d-953b-4f01-9481-6367f1f8c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.335732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-f26ef69d-953b-4f01-9481-6367f1f8c47d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:54.337470 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f26ef69d-953b-4f01-9481-6367f1f8c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.373060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:54.391951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.393198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.393716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.394223 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.395296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.397469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.412412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1157/2309] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:54.422635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.423380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.423876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.430218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1153/2310] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:54.434006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.435900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.445944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:54.447180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.450406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26cc44-3ee5-4bed-b5c0-4b3b191dd000 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.455841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.456521 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.457708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.458010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.458751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.460196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1158/2311] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:54.460398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.461104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.472689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07452a34-04e0-4edd-a123-1736b955a3c4 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.478655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.479452 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.479736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.480014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.480839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:34:54.481149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.481677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1154/2312] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:54.482795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.482910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.496384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10124d15-6d2d-4406-83bf-544b38da7225 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.499100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1159/2313] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:34:54.502472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.503097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.503217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.503474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.503910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.504683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.506397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.513980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c810b3-47bb-4adf-a7fc-6b59c4824123 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.518896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1155/2314] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:54.521447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.522122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.522698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.523018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.523202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.523678 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.525782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.534250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b313f-e591-4076-81b5-d718a6b3b5f0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.545032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600db1b-b02a-4e57-b77a-0448fabbc4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.546158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600db1b-b02a-4e57-b77a-0448fabbc4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.546437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600db1b-b02a-4e57-b77a-0448fabbc4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.546844 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600db1b-b02a-4e57-b77a-0448fabbc4a5 admin admin] PATH_INFO: `/v3/projects/98c2bede2a03450bbd854a3cca3539ab/users/5f9f36d7cf3f4d41848322530f535708/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.563121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1160/2315] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:54.563549 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.569883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:54.578179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600db1b-b02a-4e57-b77a-0448fabbc4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c2bede2a03450bbd854a3cca3539ab, user_id=5f9f36d7cf3f4d41848322530f535708, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.582719 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600db1b-b02a-4e57-b77a-0448fabbc4a5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.599806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1156/2316] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:54 2022] PUT /identity/v3/projects/98c2bede2a03450bbd854a3cca3539ab/users/5f9f36d7cf3f4d41848322530f535708/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:54.611802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.612553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.612872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.655233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db412f-54f2-4b9e-9b4f-037d710c4f81 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.655457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] MFA Rules not processed for user `5f9f36d7cf3f4d41848322530f535708`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:54.655457 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:54.736250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1157/2317] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:54.741497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.741900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.743295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.743893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.744888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.745826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.747897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.761181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3a72f-12ca-4dc7-988b-f36bf9feafc6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.781945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.783201 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.783581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.784111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.786221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.792561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1161/2318] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:54.792869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.808693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.815751 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58437f2d-7ff9-41a0-be9b-e2c95bfd5031 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.826363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.827094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.827391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.827652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.828447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.830174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.847129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1158/2319] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:34:54.851823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.858651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1162/2320] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:54.860474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:54.893814 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290208c-606e-4995-9d58-f069db756d92 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.969734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ebced-1d11-4639-92cb-951ce299fb27 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:54.974667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.975434 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.975803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.976143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.976945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.978947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:54.994425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:54.995714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:54.996481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:54.996835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:54.997155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:54.998014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:54.999825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.009018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1159/2321] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:34:54 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:34:55.014301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.039456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1163/2322] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:55.043954 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ebce0-bc26-4d30-aa17-8da3f9d717ed admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.044848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.050526 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.051224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.051506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.051785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.052531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.054626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.067753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1160/2323] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:34:55.071965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.072683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22872635-6159-447c-bc69-8ff590c65640 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.077668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.078956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.079479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.079791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] PATH_INFO: `/v3/projects/bf03ddd8a2004a0186d6286195ccca43/users/1aceb8120f9140eeb40bfa8e57d0c539/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.083663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf9e57-1f84-495d-837a-35e03b2645b5 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.089218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6c701-3b96-4fd3-940b-60a1a76f9ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.090235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6c701-3b96-4fd3-940b-60a1a76f9ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.090619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6c701-3b96-4fd3-940b-60a1a76f9ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.090979 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6c701-3b96-4fd3-940b-60a1a76f9ef3 admin admin] PATH_INFO: `/v3/projects/834a9811073d42d8ba64f256d4a78b26/users/8d7c48cee4f440da995f67e36eca634c/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.109535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf03ddd8a2004a0186d6286195ccca43, user_id=1aceb8120f9140eeb40bfa8e57d0c539, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.113140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6c701-3b96-4fd3-940b-60a1a76f9ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834a9811073d42d8ba64f256d4a78b26, user_id=8d7c48cee4f440da995f67e36eca634c, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.113266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.117423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6c701-3b96-4fd3-940b-60a1a76f9ef3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.144727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1164/2324] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:55 2022] PUT /identity/v3/projects/bf03ddd8a2004a0186d6286195ccca43/users/1aceb8120f9140eeb40bfa8e57d0c539/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:55.145061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1161/2325] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:55 2022] PUT /identity/v3/projects/834a9811073d42d8ba64f256d4a78b26/users/8d7c48cee4f440da995f67e36eca634c/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:55.145171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.147855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.148230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.148514 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.154360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:55.191912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] MFA Rules not processed for user `1aceb8120f9140eeb40bfa8e57d0c539`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:55.193656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:55.238181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913d46-5fb4-4423-86eb-1d76a23b7f52 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.278973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1162/2326] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:55.290060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.291519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:55.306106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.307201 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.307488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.307771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.308514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.310212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.322260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1165/2327] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:34:55.326841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.339955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2d96a-c9aa-4240-a8a0-293efd955a4b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.345833 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd59a-ace1-4d7d-8546-215214509a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.346938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd59a-ace1-4d7d-8546-215214509a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.347213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd59a-ace1-4d7d-8546-215214509a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.347485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd59a-ace1-4d7d-8546-215214509a46 admin admin] PATH_INFO: `/v3/projects/834a9811073d42d8ba64f256d4a78b26/users/8d7c48cee4f440da995f67e36eca634c/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.349530 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cd59a-ace1-4d7d-8546-215214509a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834a9811073d42d8ba64f256d4a78b26, user_id=8d7c48cee4f440da995f67e36eca634c, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.352192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cd59a-ace1-4d7d-8546-215214509a46 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.363900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1166/2328] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:34:55 2022] PUT /identity/v3/projects/834a9811073d42d8ba64f256d4a78b26/users/8d7c48cee4f440da995f67e36eca634c/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:55.376776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.377066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.377339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.388186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9788-c87d-4871-8a27-d1be6bde1399 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.416651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] MFA Rules not processed for user `8d7c48cee4f440da995f67e36eca634c`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:55.418220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:55.438773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:55.469206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200a524-5846-4dde-9a64-0197f2b26484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.470685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200a524-5846-4dde-9a64-0197f2b26484 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.471130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200a524-5846-4dde-9a64-0197f2b26484 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.471850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200a524-5846-4dde-9a64-0197f2b26484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.503892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1167/2329] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:55.525255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:55.526692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:55.547451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200a524-5846-4dde-9a64-0197f2b26484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.551347 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200a524-5846-4dde-9a64-0197f2b26484 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.551533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:55.559817 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1163/2330] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:55.620605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859d80d-593e-47ca-832e-ca5c65d6d228 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:55.631046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0c60d-f255-4691-977d-e55df84aeb20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:55.631927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0c60d-f255-4691-977d-e55df84aeb20 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.632213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0c60d-f255-4691-977d-e55df84aeb20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.632484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0c60d-f255-4691-977d-e55df84aeb20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.693796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0c60d-f255-4691-977d-e55df84aeb20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:55.695273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0c60d-f255-4691-977d-e55df84aeb20 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:55.706498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1168/2331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:34:55.844152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:55.844590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:55.845127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:55.866432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] MFA Rules not processed for user `1fc65fea19f94fd18d83ad8ce3bbb476`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:55.867809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:55.952017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1164/2332] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:58.390691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:58.391288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:58.391288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:58.413266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:58.416093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:58.480607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1169/2333] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:58.493134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:58.494582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:58.524801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:58.577440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa110260-9690-4090-a932-b57993a8a5cc None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:58.583673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526177e7-dae9-499c-b351-886dc16b7ec8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:58.584348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526177e7-dae9-499c-b351-886dc16b7ec8 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:58.584657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526177e7-dae9-499c-b351-886dc16b7ec8 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:58.585070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526177e7-dae9-499c-b351-886dc16b7ec8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:58.594060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526177e7-dae9-499c-b351-886dc16b7ec8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:58.595769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526177e7-dae9-499c-b351-886dc16b7ec8 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:58.604351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1165/2334] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:34:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:34:58.606710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4c91-2828-48fb-9b53-e6e1b68c78bc None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:58.611964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:58.613634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:58.613634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:58.613634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] PATH_INFO: `/v3/users/5f9f36d7cf3f4d41848322530f535708` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:58.616595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f9f36d7cf3f4d41848322530f535708)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:58.619021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:58.646998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:58.684232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f9f36d7cf3f4d41848322530f535708'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:58.685900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f9f36d7cf3f4d41848322530f535708'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:58.702789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f9f36d7cf3f4d41848322530f535708'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:58.703737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f9f36d7cf3f4d41848322530f535708'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:58.717087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f9f36d7cf3f4d41848322530f535708'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:58.766523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1170/2335] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:58 2022] DELETE /identity/v3/users/5f9f36d7cf3f4d41848322530f535708 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:34:58.768062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:58.768353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:58.768651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:58.818188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:58.819951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:58.851927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:58.872216 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:58.874188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:58.953158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1166/2336] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:34:58.967381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:58.970980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:59.050755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cdb86-21d9-4493-bb28-3663a11f9ed9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:59.068926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:59.071706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:59.071995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:59.072258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] PATH_INFO: `/v3/users/1aceb8120f9140eeb40bfa8e57d0c539` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:59.076851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aceb8120f9140eeb40bfa8e57d0c539)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:59.082877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:59.094740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3bcb6-e9a1-4a09-b085-ac04241c5a17 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:59.101818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed45ea-13c9-4387-85a1-5a5222673573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:59.102548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed45ea-13c9-4387-85a1-5a5222673573 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:59.102829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed45ea-13c9-4387-85a1-5a5222673573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:59.103144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed45ea-13c9-4387-85a1-5a5222673573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:59.130577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed45ea-13c9-4387-85a1-5a5222673573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:59.133049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed45ea-13c9-4387-85a1-5a5222673573 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:59.151994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1167/2337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:59.261728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aceb8120f9140eeb40bfa8e57d0c539'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.262738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aceb8120f9140eeb40bfa8e57d0c539'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.292859 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aceb8120f9140eeb40bfa8e57d0c539'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.293973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aceb8120f9140eeb40bfa8e57d0c539'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.317648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aceb8120f9140eeb40bfa8e57d0c539'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.343571 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1171/2338] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:58 2022] DELETE /identity/v3/users/1aceb8120f9140eeb40bfa8e57d0c539 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:59.348795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:59.349580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:59.350034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:59.389602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:34:59.392545 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:59.657676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1168/2339] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:34:59.673259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:59.676207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:59.718053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:34:59.720862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:34:59.752226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:59.754658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88cdc0-c420-4fdb-80bf-4c3d8d2ea2cb None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:59.762431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:59.765280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:59.765919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:59.767087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] PATH_INFO: `/v3/users/272fff49c5dc4db6a19d7bc6a6bf6bf6` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:59.769408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=272fff49c5dc4db6a19d7bc6a6bf6bf6)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:59.772065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:59.835940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498171-2763-48c1-a7fe-0154040e0bfb admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:34:59.840691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddd4b6-5730-468d-a6f7-a4ca58371850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:34:59.841521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd4b6-5730-468d-a6f7-a4ca58371850 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:59.855872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd4b6-5730-468d-a6f7-a4ca58371850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:59.856480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd4b6-5730-468d-a6f7-a4ca58371850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:34:59.861828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272fff49c5dc4db6a19d7bc6a6bf6bf6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.862785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272fff49c5dc4db6a19d7bc6a6bf6bf6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.876754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:34:59.882566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddd4b6-5730-468d-a6f7-a4ca58371850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:34:59.884849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddd4b6-5730-468d-a6f7-a4ca58371850 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:34:59.888485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272fff49c5dc4db6a19d7bc6a6bf6bf6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.889574 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272fff49c5dc4db6a19d7bc6a6bf6bf6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.900974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1172/2340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:34:59.949229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272fff49c5dc4db6a19d7bc6a6bf6bf6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:34:59.982443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1169/2341] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:34:59 2022] DELETE /identity/v3/users/272fff49c5dc4db6a19d7bc6a6bf6bf6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:34:59.990645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:34:59.990918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:34:59.991188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:00.024943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:00.030415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:00.250924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1173/2342] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:00.266492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:00.268393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:00.414992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2082e-c750-4315-9edb-dbdaa25479fb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:00.423406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:00.424455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:00.424910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:00.425193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:00.438310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:00.440269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:00.494905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1170/2343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:00.520360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:00.521909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:00.553459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fd60f-02fa-43be-ba71-8e6b92f602ec None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:00.568225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:00.569160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:00.569479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:00.569702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] PATH_INFO: `/v3/projects/98c2bede2a03450bbd854a3cca3539ab` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:00.570925 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c2bede2a03450bbd854a3cca3539ab)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:00.572885 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:00.699875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c2bede2a03450bbd854a3cca3539ab', 'request_id': 'req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:00.701129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:00.702888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:00.715158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c2bede2a03450bbd854a3cca3539ab', 'request_id': 'req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:00.730363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c2bede2a03450bbd854a3cca3539ab', 'request_id': 'req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:00.744484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b463e1-e84b-4f05-99a4-52c55d783ec3 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:00.754813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] The token cache is being invalidate because project 98c2bede2a03450bbd854a3cca3539ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:00.755102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:00.756539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:00.762441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1174/2344] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:00 2022] DELETE /identity/v3/projects/98c2bede2a03450bbd854a3cca3539ab => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:00.767261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:00.771321 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:00.972344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:01.034529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:01.036182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.038006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.040609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] PATH_INFO: `/v3/projects/bf03ddd8a2004a0186d6286195ccca43` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.042820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf03ddd8a2004a0186d6286195ccca43)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:01.044025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c3886-a0e2-4e2f-a305-ee9a3a64610b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:01.049540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:01.065663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:01.069278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.071900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.072285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] PATH_INFO: `/v3/users/8d7c48cee4f440da995f67e36eca634c` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.074365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7c48cee4f440da995f67e36eca634c)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:01.078601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:01.162496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf03ddd8a2004a0186d6286195ccca43', 'request_id': 'req-2592cd21-2d7d-4f5f-8519-5e43042e8178'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.178919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf03ddd8a2004a0186d6286195ccca43', 'request_id': 'req-2592cd21-2d7d-4f5f-8519-5e43042e8178'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.204504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf03ddd8a2004a0186d6286195ccca43', 'request_id': 'req-2592cd21-2d7d-4f5f-8519-5e43042e8178'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.223612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] The token cache is being invalidate because project bf03ddd8a2004a0186d6286195ccca43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:01.224021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.225014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-2592cd21-2d7d-4f5f-8519-5e43042e8178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.227966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1171/2345] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:00 2022] DELETE /identity/v3/projects/bf03ddd8a2004a0186d6286195ccca43 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:01.236893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf9082-1825-406f-8486-4252962ebf23 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.237896 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf9082-1825-406f-8486-4252962ebf23 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.238827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf9082-1825-406f-8486-4252962ebf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.272790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-50bf9082-1825-406f-8486-4252962ebf23 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:01.274837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7c48cee4f440da995f67e36eca634c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.274959 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-50bf9082-1825-406f-8486-4252962ebf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:01.275832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7c48cee4f440da995f67e36eca634c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.301135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:01.322936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7c48cee4f440da995f67e36eca634c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.326131 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7c48cee4f440da995f67e36eca634c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.343551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7c48cee4f440da995f67e36eca634c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.402083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1172/2346] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:01.411164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.411515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.411909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.418167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1175/2347] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:00 2022] DELETE /identity/v3/users/8d7c48cee4f440da995f67e36eca634c => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:01.445394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:01.446700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:01.561541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1173/2348] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:35:01.565737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:01.580191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cf3ba-5fdb-4263-8b15-bd480a829507 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:01.600030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:01.601550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.604001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.604366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.605287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:01.605695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:01.612940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:01.649173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:01.650888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:01.653496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1176/2349] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:35:01.658033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:01.674869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029e347-d851-4a8c-8592-67f631bab78e None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:01.682958 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:01.683945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.684409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.684787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] PATH_INFO: `/v3/projects/617b54d74adc4dc1add9ea3bad81282d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.686231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=617b54d74adc4dc1add9ea3bad81282d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:01.689033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:01.689290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad921387-ae0d-4645-95b3-0ee06dadd7a0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:01.712406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:01.713893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:01.714222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:01.714501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:01.717018 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:01.734338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:01.815523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1177/2350] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:01 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:01.820231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:01.829826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617b54d74adc4dc1add9ea3bad81282d', 'request_id': 'req-d575e763-4bf7-4661-a9a3-0263a7dcab87'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.844373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617b54d74adc4dc1add9ea3bad81282d', 'request_id': 'req-d575e763-4bf7-4661-a9a3-0263a7dcab87'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.852192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617b54d74adc4dc1add9ea3bad81282d', 'request_id': 'req-d575e763-4bf7-4661-a9a3-0263a7dcab87'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.882270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] The token cache is being invalidate because project 617b54d74adc4dc1add9ea3bad81282d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:01.882703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.883273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:01.885905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1174/2351] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:01 2022] DELETE /identity/v3/projects/617b54d74adc4dc1add9ea3bad81282d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:01.893110 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:01.894790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.006105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36ef77-0394-4f5b-b72c-6bb455562c8d admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.020780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:02.022877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d42b62-8e73-4270-9751-b6aba1920576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.023592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d42b62-8e73-4270-9751-b6aba1920576 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.023876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d42b62-8e73-4270-9751-b6aba1920576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.024181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d42b62-8e73-4270-9751-b6aba1920576 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.025032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d42b62-8e73-4270-9751-b6aba1920576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.027489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d42b62-8e73-4270-9751-b6aba1920576 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.082617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575e763-4bf7-4661-a9a3-0263a7dcab87 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.088034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1178/2352] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:01 2022] POST /identity/v3/users => generated 287 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:35:02.092267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.092508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.092663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.092948 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.093563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.093863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.094348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] PATH_INFO: `/v3/users/01fa74b7c1ee418fa9938f1f9f6bebc4` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.095509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fa74b7c1ee418fa9938f1f9f6bebc4)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.097533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.144456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:02.146342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.168359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1179/2353] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:02.178992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.182625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.198401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fa74b7c1ee418fa9938f1f9f6bebc4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.199218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fa74b7c1ee418fa9938f1f9f6bebc4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.202831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fed85-d73a-4f30-b97a-972837f88eec None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.208550 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16364471-551f-48bd-80a7-da7e6f3bf19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.209311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364471-551f-48bd-80a7-da7e6f3bf19f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.209518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364471-551f-48bd-80a7-da7e6f3bf19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.212614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364471-551f-48bd-80a7-da7e6f3bf19f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.212614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16364471-551f-48bd-80a7-da7e6f3bf19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.212614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16364471-551f-48bd-80a7-da7e6f3bf19f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.216203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:02.226716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1180/2354] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:02.234969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.235422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.236159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fa74b7c1ee418fa9938f1f9f6bebc4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.236956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fa74b7c1ee418fa9938f1f9f6bebc4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.239142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.253016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fa74b7c1ee418fa9938f1f9f6bebc4'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.262612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:02.264024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.273570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1181/2355] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:02.275166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.276524 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.284766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1175/2356] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:01 2022] DELETE /identity/v3/users/01fa74b7c1ee418fa9938f1f9f6bebc4 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:02.286170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.289121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.294630 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa459bc-ec80-4b77-96bb-f7f30437f704 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.378761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.379975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.380340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.380619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] PATH_INFO: `/v3/projects/834a9811073d42d8ba64f256d4a78b26` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.382689 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834a9811073d42d8ba64f256d4a78b26)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.386296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.426253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1476ddb-219f-4540-9d7c-de6b00a8a750 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.452252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.453637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.453927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.455590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.456362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.460676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.487161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834a9811073d42d8ba64f256d4a78b26', 'request_id': 'req-61344af5-533d-4f3e-95eb-a1e14ef57af7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.491945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1176/2357] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:02.499493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.500954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834a9811073d42d8ba64f256d4a78b26', 'request_id': 'req-61344af5-533d-4f3e-95eb-a1e14ef57af7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.519569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834a9811073d42d8ba64f256d4a78b26', 'request_id': 'req-61344af5-533d-4f3e-95eb-a1e14ef57af7'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.528420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e08f33-912a-4528-ad51-fe995d29007e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.558184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] The token cache is being invalidate because project 834a9811073d42d8ba64f256d4a78b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:02.558526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.560371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-61344af5-533d-4f3e-95eb-a1e14ef57af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:02.562654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1182/2358] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:02 2022] DELETE /identity/v3/projects/834a9811073d42d8ba64f256d4a78b26 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:02.572158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.572486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.572797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.585622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:02.609270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:02.610649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.623983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.625219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.625843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.626344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.627560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:02.628462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.630751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a88c63-863c-4f97-bca8-71fd4c9e057d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.640513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1183/2359] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:02.644975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.648141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.657860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1177/2360] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:35:02.659368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.663703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae64d6-263c-4938-bb79-a97856a67f34 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.671865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.673450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.673797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.674016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.674427 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.674675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] PATH_INFO: `/v3/projects/1f38708a09ce4c8d898c52c85df12a5a/users/7b75dc3d26fe4bc292868b63431735db/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.696303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f38708a09ce4c8d898c52c85df12a5a, user_id=7b75dc3d26fe4bc292868b63431735db, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.699124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.706858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:02.706858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.720307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1178/2361] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:02.724285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1184/2362] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:02 2022] PUT /identity/v3/projects/1f38708a09ce4c8d898c52c85df12a5a/users/7b75dc3d26fe4bc292868b63431735db/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:02.724809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.724910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:02.726188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.731623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.814656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:02.854363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec28996-fd22-4746-82c3-a962e19cf654 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.857184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92605d9c-c06c-4c44-95a3-bf33e7bbc5ae None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:02.861518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314b0f6-10bf-454b-bbc0-d7e03c4917a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.862377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b0f6-10bf-454b-bbc0-d7e03c4917a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.862666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b0f6-10bf-454b-bbc0-d7e03c4917a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.862941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b0f6-10bf-454b-bbc0-d7e03c4917a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.863646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314b0f6-10bf-454b-bbc0-d7e03c4917a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.865746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314b0f6-10bf-454b-bbc0-d7e03c4917a3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.921167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1185/2363] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:02.922504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.923316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.923316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.937927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d0d4-c218-4602-9535-4e65aaa7f848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:02.938671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d0d4-c218-4602-9535-4e65aaa7f848 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.938960 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d0d4-c218-4602-9535-4e65aaa7f848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.939242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d0d4-c218-4602-9535-4e65aaa7f848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:02.956691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:02.957066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0d0d4-c218-4602-9535-4e65aaa7f848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:02.957170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:02.957279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0d0d4-c218-4602-9535-4e65aaa7f848 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:02.968384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1179/2364] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:02.970661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:02.971209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:02.971598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.009028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:03.011430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:03.106481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1186/2365] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:03.107763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.132700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0fd8-62eb-4e51-9a49-693fd33db721 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.145165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.145922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.146355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.146654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.147406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.150203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.156388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1180/2366] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:03.162355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.167515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:03.195905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1187/2367] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:03 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:03.196992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.225187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7c6d5-52d3-4415-b09e-ca3b03c571c4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.230860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.231779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.232221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.232541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.233326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:03.233714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.235399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.241392 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:03.250471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1188/2368] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:35:03.251361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.263098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802155-67e6-4dc9-9003-a7e3e8174bdf admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.282853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.283637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.284832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.285110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.301752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.303491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.315407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdeaff-1905-4b3a-981f-5e6cb5175ada None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.320441 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1189/2369] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:03.324233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.325264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.325861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.326395 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.330006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.341624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.343839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.354993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2159-8fd7-4b57-b1aa-89389b5adcd5 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.365032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1181/2370] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:03.366034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.367591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.368092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.368604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.369637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.370802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.374847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.384956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be252f-aeca-4d6a-9734-7826a589c343 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.391020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.392143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.392531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.393323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.394315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.397014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.409247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1182/2371] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:03.423649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.430530 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1190/2372] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:03 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:03.435420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.436033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318fa7-ad7f-4f26-ba7c-b1883d83ad73 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.512728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383080-36f2-4e45-ac2c-79865c75bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.513732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383080-36f2-4e45-ac2c-79865c75bfb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.514117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383080-36f2-4e45-ac2c-79865c75bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.514486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383080-36f2-4e45-ac2c-79865c75bfb3 admin admin] PATH_INFO: `/v3/projects/c7b272349089480984fd1c5eabe6dce9/users/72419918d6c34080b6a4a4c73b2c06b2/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.535842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383080-36f2-4e45-ac2c-79865c75bfb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b272349089480984fd1c5eabe6dce9, user_id=72419918d6c34080b6a4a4c73b2c06b2, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.553072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383080-36f2-4e45-ac2c-79865c75bfb3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.555565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddb04e-4220-4774-bed6-de25b54820cc admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.562546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.563498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.563950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.564310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.565291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.576869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1183/2373] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:03 2022] PUT /identity/v3/projects/c7b272349089480984fd1c5eabe6dce9/users/72419918d6c34080b6a4a4c73b2c06b2/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:03.580597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.583098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.583570 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.583964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.622837 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:03.625693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:03.635868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1191/2374] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:03 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:03.637127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.640744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:03.654058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:03.674881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d927a4c-ec25-43f4-b24a-37c34982bd49 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.723439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1184/2375] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:03.724679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.749133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a2c9b-0631-42d8-b712-55212605245d None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.764573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.766003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.766335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.766653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.777715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.779397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.804862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1185/2376] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:03.804862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:03.841446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.842430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.842722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.843802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] PATH_INFO: `/v3/projects/a338f94294fe4a00b2921c62fdb0ad85` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.845701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a338f94294fe4a00b2921c62fdb0ad85)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.850404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:03.913332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:03.942464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ea00f-f52e-4a06-9684-011a5bc3303c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:03.946633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:03.947239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:03.947524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:03.947796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:03.995816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:03.997336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.000126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a338f94294fe4a00b2921c62fdb0ad85', 'request_id': 'req-a46da3a9-fe3a-45d3-92cb-b9297e75f090'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:04.018742 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a338f94294fe4a00b2921c62fdb0ad85', 'request_id': 'req-a46da3a9-fe3a-45d3-92cb-b9297e75f090'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:04.020820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1186/2377] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:04.021905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.033123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3828c7-d91b-46d9-96d5-41ede1391e35 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.038870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a338f94294fe4a00b2921c62fdb0ad85', 'request_id': 'req-a46da3a9-fe3a-45d3-92cb-b9297e75f090'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:04.063608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] The token cache is being invalidate because project a338f94294fe4a00b2921c62fdb0ad85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:04.064302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:04.065391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-a46da3a9-fe3a-45d3-92cb-b9297e75f090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:04.068656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1192/2378] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:03 2022] DELETE /identity/v3/projects/a338f94294fe4a00b2921c62fdb0ad85 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:04.073741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842acb56-ff4a-4e7b-99ab-aa45f2e3f3e0 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.074084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842acb56-ff4a-4e7b-99ab-aa45f2e3f3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.074360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842acb56-ff4a-4e7b-99ab-aa45f2e3f3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.099702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-842acb56-ff4a-4e7b-99ab-aa45f2e3f3e0 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:04.101238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-842acb56-ff4a-4e7b-99ab-aa45f2e3f3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:04.183046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1193/2379] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:04.193683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.193987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.194264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.242114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:04.243484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:04.258233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.259439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.259677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1194/2380] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:04.260876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.261356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.263194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.263636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.270113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.293750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1187/2381] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:04.298582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.301008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:04.304063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a2f6a-fc9e-4a84-ba84-f515dd920280 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.325490 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.326200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.326487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.326766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.327588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:04.327909 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.329602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.345310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a759e22-7d62-4dd9-83b5-8f5f2c846385 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.359896 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1195/2382] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:35:04.360895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.381098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b040943-2898-45ab-a8ec-27bd93b583db admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.385167 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b2c59-51c0-46cf-8986-c09191f646ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.386723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b2c59-51c0-46cf-8986-c09191f646ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.387262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b2c59-51c0-46cf-8986-c09191f646ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.387748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b2c59-51c0-46cf-8986-c09191f646ed admin admin] PATH_INFO: `/v3/projects/3bd7e4eb5c9240388d7e8744afdecb3d/users/3abffceec7bb45eba8ae593903d6ac85/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.395411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.396105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.396389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.396695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.397413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.400952 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.414449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:04.441890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1196/2383] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:04 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:04.447933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.452098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b2c59-51c0-46cf-8986-c09191f646ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd7e4eb5c9240388d7e8744afdecb3d, user_id=3abffceec7bb45eba8ae593903d6ac85, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.464274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b2c59-51c0-46cf-8986-c09191f646ed admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.523206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1188/2384] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:04 2022] PUT /identity/v3/projects/3bd7e4eb5c9240388d7e8744afdecb3d/users/3abffceec7bb45eba8ae593903d6ac85/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:04.537794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.538144 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.538411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.575209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] MFA Rules not processed for user `3abffceec7bb45eba8ae593903d6ac85`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:04.577589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:04.601250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894f91f-f34e-4388-a1fa-8c7f584d1543 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.737281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1189/2385] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:04.738725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.742445 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:04.806442 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.808338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.808643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.808922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.811000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.817478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.869572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:04.892396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1197/2386] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:04 2022] POST /identity/v3/users => generated 286 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:04.895984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.908914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c790-7433-40f3-85a5-b66e533d0b65 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.915790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c88dc-3202-49ff-b9a3-040d816dc953 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.922920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.924917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.925407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.925609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.926310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.926593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.926865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.927459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.927627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.929918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.950242 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.952258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:04.952668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1198/2387] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:04.956548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:04.970590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb437-e8bd-45fa-b847-5a2d739a89fd admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:04.976854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1190/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:04.979971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ba91-8dab-447a-9544-1adc837a9106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:04.980907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ba91-8dab-447a-9544-1adc837a9106 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:04.981982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ba91-8dab-447a-9544-1adc837a9106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:04.982801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ba91-8dab-447a-9544-1adc837a9106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:04.989186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76ba91-8dab-447a-9544-1adc837a9106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:04.992213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76ba91-8dab-447a-9544-1adc837a9106 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:05.015315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1199/2389] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:35:05.020277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:05.037689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9a28-9f70-4596-a894-8452c26fff83 service glance] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:05.048147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:05.049461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:05.049842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:05.050259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] PATH_INFO: `/v3/projects/e8abe616c14a4920b1cbc45733fa5771/users/e81aa9d47cd1470abef31a27839eaf86/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:05.085067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8abe616c14a4920b1cbc45733fa5771, user_id=e81aa9d47cd1470abef31a27839eaf86, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:05.088947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:05.141549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1191/2390] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:05 2022] PUT /identity/v3/projects/e8abe616c14a4920b1cbc45733fa5771/users/e81aa9d47cd1470abef31a27839eaf86/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:05.147044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b218f-a7b3-4873-882c-01e70374ea1f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:05.147611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b218f-a7b3-4873-882c-01e70374ea1f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:05.148116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b218f-a7b3-4873-882c-01e70374ea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:05.189755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a54b218f-a7b3-4873-882c-01e70374ea1f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:05.191173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a54b218f-a7b3-4873-882c-01e70374ea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:05.344127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1200/2391] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:05.354975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:05.356815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:05.529143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c066b3-700f-48c8-bbdd-cd5a2886ad7b admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:05.543713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:05.545119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:05.545549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:05.545880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:05.559631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:05.561329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:05.575919 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1192/2392] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:05.689475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:05.690105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:05.690577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:05.706279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:05.766038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] MFA Rules not processed for user `7b75dc3d26fe4bc292868b63431735db`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:05.771568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:05.986615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1201/2393] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:05.991066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:05.998108 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:06.028503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb358e-216e-4d72-b5df-21f8934ee396 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:06.045281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:06.046236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:06.046774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:06.047152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:06.048055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:06.050580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:06.107610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1193/2394] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:05 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:06.108928 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:06.135448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96582a3-d619-4bcd-a0be-5dd7a1363e5a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:06.274572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:06.275712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:06.276202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:06.277473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:06.278532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:06.281235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:06.325494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1202/2395] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:06 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:06.328547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:06.343511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312c310-70bb-49fa-a92e-3ec14c0ee5de admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:06.358245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:06.359403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:06.359898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:06.360192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:06.361246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:06.363669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:06.399910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1194/2396] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:06.401119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:06.419818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c199b1-417a-4655-9a28-3e2d78eda821 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:06.447808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:06.448575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:06.452855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:06.456250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] PATH_INFO: `/v3/projects/e7186bcef7054f14b10d8ae76e133857/users/fa5d1b7938f8489c82b8420efdb34830/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:06.502053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7186bcef7054f14b10d8ae76e133857, user_id=fa5d1b7938f8489c82b8420efdb34830, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:06.504900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:06.572986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1203/2397] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:06 2022] PUT /identity/v3/projects/e7186bcef7054f14b10d8ae76e133857/users/fa5d1b7938f8489c82b8420efdb34830/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:06.577398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:06.590811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7684dd-bd55-434a-af60-5a4736e7b100 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:06.828901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b5ab3-f2c4-45cf-9d60-dbde07276b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:06.833945 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5ab3-f2c4-45cf-9d60-dbde07276b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:06.835217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5ab3-f2c4-45cf-9d60-dbde07276b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:06.835538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b5ab3-f2c4-45cf-9d60-dbde07276b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:06.836306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b5ab3-f2c4-45cf-9d60-dbde07276b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:06.838850 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b5ab3-f2c4-45cf-9d60-dbde07276b04 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:06.872066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1195/2398] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:06.872294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:06.890708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cf06b-1d17-4cb6-8590-1fa5a754ea81 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:06.896011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:06.896994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:06.897279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:06.897712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] PATH_INFO: `/v3/projects/e7186bcef7054f14b10d8ae76e133857/users/fa5d1b7938f8489c82b8420efdb34830/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:06.900761 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7186bcef7054f14b10d8ae76e133857, user_id=fa5d1b7938f8489c82b8420efdb34830, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:06.909531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:06.931669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1204/2399] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:06 2022] PUT /identity/v3/projects/e7186bcef7054f14b10d8ae76e133857/users/fa5d1b7938f8489c82b8420efdb34830/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:10.499722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:10.500318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:10.500424 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:10.573978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] MFA Rules not processed for user `fa5d1b7938f8489c82b8420efdb34830`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:10.575663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:10.629659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:10.748855 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1196/2400] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:10.763911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:10.767314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:10.857946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:10.867629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd0ef1-12ba-44bf-a52c-b5ca8946fbed admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:10.885014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6b2a2-55cb-4765-8163-3269fd091820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:10.887183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6b2a2-55cb-4765-8163-3269fd091820 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:10.893300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6b2a2-55cb-4765-8163-3269fd091820 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:10.894664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6b2a2-55cb-4765-8163-3269fd091820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:11.111610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6b2a2-55cb-4765-8163-3269fd091820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:11.113074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6b2a2-55cb-4765-8163-3269fd091820 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:11.149407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1205/2401] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:35:12.954142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:12.967965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32604f82-43eb-4fb0-b004-1044208cd6ed None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:12.973387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:12.974288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:12.974709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:12.975131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:12.989303 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:12.991924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:13.021663 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1197/2402] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:14.114807 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:14.116277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:14.116765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:14.150373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] MFA Rules not processed for user `72419918d6c34080b6a4a4c73b2c06b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:14.151915 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:14.283328 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1206/2403] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:14.288234 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:14.290331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:14.439283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1d0f3-bf62-4e72-ab71-fcbe85a65594 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:14.451749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:14.452446 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:14.452741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:14.453017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:14.453731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:14.455736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:14.512575 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1198/2404] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:14 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:35:14.516703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:14.537791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b5a53-20f6-450e-aadf-0f8a7df9bdf3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:14.665352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:14.666129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:14.669185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:14.669560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:14.670390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:14.675355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:14.755602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1207/2405] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:14 2022] POST /identity/v3/users => generated 295 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:14.758344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:14.781316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486bd6-9ee6-4c84-ab3e-4d3fd90fc124 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:14.800104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:14.801310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:14.802613 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:14.803090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:14.804102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:14.808644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:14.830589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1199/2406] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:14.836602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:14.858020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac00a5-50ab-49a8-bf3b-1afbb7eb195c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:14.864687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:14.866137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:14.866675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:14.867169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] PATH_INFO: `/v3/projects/15962efa920b497b88991a1d1f1b26c6/users/ca9f038b657a4328869a9c3499d7ea1d/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:14.919769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] RBAC: Authorizing `identity:create_grant(project_id=15962efa920b497b88991a1d1f1b26c6, user_id=ca9f038b657a4328869a9c3499d7ea1d, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:14.925871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:14.948730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:14.961207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1208/2407] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:14 2022] PUT /identity/v3/projects/15962efa920b497b88991a1d1f1b26c6/users/ca9f038b657a4328869a9c3499d7ea1d/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:14.966203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:14.987681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee08bce-5122-42c2-a6c7-82b2f97b5434 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:15.220043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d22d53-d3a2-45cd-a20b-08e41da6425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:15.230432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d22d53-d3a2-45cd-a20b-08e41da6425a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:15.230903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d22d53-d3a2-45cd-a20b-08e41da6425a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:15.231330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d22d53-d3a2-45cd-a20b-08e41da6425a admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:15.232255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d22d53-d3a2-45cd-a20b-08e41da6425a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:15.241351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d22d53-d3a2-45cd-a20b-08e41da6425a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:15.271573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1200/2408] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:15.274799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:15.288591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d22a1f-2659-40b7-addc-c5fc931e904c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:15.298568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:15.299606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:15.299970 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:15.300293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] PATH_INFO: `/v3/projects/15962efa920b497b88991a1d1f1b26c6/users/ca9f038b657a4328869a9c3499d7ea1d/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:15.313836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=15962efa920b497b88991a1d1f1b26c6, user_id=ca9f038b657a4328869a9c3499d7ea1d, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:15.318424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:15.348004 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1209/2409] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:15 2022] PUT /identity/v3/projects/15962efa920b497b88991a1d1f1b26c6/users/ca9f038b657a4328869a9c3499d7ea1d/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:16.460969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:16.461532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:16.461628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:16.494104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:16.550401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] MFA Rules not processed for user `e81aa9d47cd1470abef31a27839eaf86`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:16.551801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:16.730622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1201/2410] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:16.735871 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:17.025068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572a28d-2e11-40d8-8d58-256539d22acd admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:17.037893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:17.039879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:17.041101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:17.041512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:17.042329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:17.047766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:17.165886 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:17.168159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1210/2411] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:16 2022] POST /identity/v3/projects => generated 373 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:17.181942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebd7d6-0007-4b82-8719-4bc53c83aa63 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:17.274833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:17.276089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:17.276818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:17.277317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:17.279013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:17.283383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:17.323969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:17.333738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1202/2412] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:17 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:35:17.337929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c8f52-1ceb-4cce-9238-a00c03040742 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:17.344339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:17.345485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:17.345782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:17.346060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:17.347234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:17.349845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:17.366634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:17.374461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1211/2413] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:17.383700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95ce6-46ce-4dca-9e0a-9da6211b25cc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:17.392940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:17.393591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:17.394301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:17.394755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] PATH_INFO: `/v3/projects/401bb3c5eb1b49eb8f2aed9166673be4/users/22c36e462dda4424b81718b0146021a6/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:17.416211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] RBAC: Authorizing `identity:create_grant(project_id=401bb3c5eb1b49eb8f2aed9166673be4, user_id=22c36e462dda4424b81718b0146021a6, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:17.421005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:17.454262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1203/2414] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:17 2022] PUT /identity/v3/projects/401bb3c5eb1b49eb8f2aed9166673be4/users/22c36e462dda4424b81718b0146021a6/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:17.458256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:17.471043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ccd90-5ded-4213-aca6-690354b3f6e4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:17.554034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba60f0-b31c-4dc6-a613-2438cab4a4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:17.554717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba60f0-b31c-4dc6-a613-2438cab4a4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:17.555405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba60f0-b31c-4dc6-a613-2438cab4a4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:17.555405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba60f0-b31c-4dc6-a613-2438cab4a4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:17.555973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba60f0-b31c-4dc6-a613-2438cab4a4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:17.557966 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba60f0-b31c-4dc6-a613-2438cab4a4f8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:17.569094 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1212/2415] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:17.573158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:17.585511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2a355-29ed-4039-a5d5-8d34cd51442b admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:17.592556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5e3c8-9bd4-40d2-ac87-e22ded48752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:17.593773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5e3c8-9bd4-40d2-ac87-e22ded48752e admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:17.594250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5e3c8-9bd4-40d2-ac87-e22ded48752e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:17.594786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5e3c8-9bd4-40d2-ac87-e22ded48752e admin admin] PATH_INFO: `/v3/projects/401bb3c5eb1b49eb8f2aed9166673be4/users/22c36e462dda4424b81718b0146021a6/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:17.597297 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5e3c8-9bd4-40d2-ac87-e22ded48752e admin admin] RBAC: Authorizing `identity:create_grant(project_id=401bb3c5eb1b49eb8f2aed9166673be4, user_id=22c36e462dda4424b81718b0146021a6, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:17.600566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5e3c8-9bd4-40d2-ac87-e22ded48752e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:17.616244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1204/2416] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:17 2022] PUT /identity/v3/projects/401bb3c5eb1b49eb8f2aed9166673be4/users/22c36e462dda4424b81718b0146021a6/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:26.697386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:26.697972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:26.697972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:26.704943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:26.740860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] MFA Rules not processed for user `22c36e462dda4424b81718b0146021a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:26.742175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:26.903567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1213/2417] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:35:27.037529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:27.038339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:27.038768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:27.085156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] MFA Rules not processed for user `ca9f038b657a4328869a9c3499d7ea1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:27.087314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:27.102254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:27.106016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:27.148268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:27.314477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1205/2418] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:27.317164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:27.320038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d523c8-8387-4d40-a8a6-2b1c0051fa9d None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:27.332975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04332be-57c5-4549-9262-db347c93c600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:27.333695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:27.334141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04332be-57c5-4549-9262-db347c93c600 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:27.334617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04332be-57c5-4549-9262-db347c93c600 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:27.335688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04332be-57c5-4549-9262-db347c93c600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:27.367066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc634-8493-4cc2-8886-ff80ccade773 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:27.378541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b001c40-3925-465d-8c1d-9c988bdaf5cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:27.382971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b001c40-3925-465d-8c1d-9c988bdaf5cc service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:27.383282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b001c40-3925-465d-8c1d-9c988bdaf5cc service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:27.383565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b001c40-3925-465d-8c1d-9c988bdaf5cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:27.432184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04332be-57c5-4549-9262-db347c93c600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:27.433844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04332be-57c5-4549-9262-db347c93c600 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:27.483790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1214/2419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:27.484552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b001c40-3925-465d-8c1d-9c988bdaf5cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:27.487335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b001c40-3925-465d-8c1d-9c988bdaf5cc service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:27.541796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1206/2420] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:28.887793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:28.889471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:28.890010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:28.928503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] MFA Rules not processed for user `26807e5e7aab425eb28e781289794265`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:28.930494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:28.937350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:29.052830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1215/2421] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:30.555811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:30.556352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:30.556544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:30.595060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] MFA Rules not processed for user `e81aa9d47cd1470abef31a27839eaf86`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:30.596808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:30.610202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1207/2422] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:33.035954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:33.113897 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727ec5-1e0c-458f-bf09-2ba1feaf42fd None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:33.121213 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:33.122627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:33.123245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:33.123632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] PATH_INFO: `/v3/users/7b75dc3d26fe4bc292868b63431735db` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:33.125274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b75dc3d26fe4bc292868b63431735db)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:33.127260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:33.201077 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b75dc3d26fe4bc292868b63431735db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.201924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b75dc3d26fe4bc292868b63431735db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.215403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b75dc3d26fe4bc292868b63431735db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.216333 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b75dc3d26fe4bc292868b63431735db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.225874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b75dc3d26fe4bc292868b63431735db'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.243702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1216/2423] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:33 2022] DELETE /identity/v3/users/7b75dc3d26fe4bc292868b63431735db => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:33.521738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:33.542455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41da1f-45bb-4521-9791-60ec26d892e3 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:33.626444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:33.627468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:33.628493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:33.628868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] PATH_INFO: `/v3/projects/1f38708a09ce4c8d898c52c85df12a5a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:33.630191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f38708a09ce4c8d898c52c85df12a5a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:33.632270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:33.689794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:33.707774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f38708a09ce4c8d898c52c85df12a5a', 'request_id': 'req-f49de463-287c-4a16-9f6a-3d5057d9af40'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.718185 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f38708a09ce4c8d898c52c85df12a5a', 'request_id': 'req-f49de463-287c-4a16-9f6a-3d5057d9af40'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.726807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f38708a09ce4c8d898c52c85df12a5a', 'request_id': 'req-f49de463-287c-4a16-9f6a-3d5057d9af40'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.741794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] The token cache is being invalidate because project 1f38708a09ce4c8d898c52c85df12a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:33.742104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.743444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.745694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1208/2424] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:33 2022] DELETE /identity/v3/projects/1f38708a09ce4c8d898c52c85df12a5a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:33.755460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:33.757680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:33.774025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e758-d4e5-4092-bfa2-22d7e4aa201a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:33.812737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:33.850626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:33.851921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:33.852388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:33.852881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] PATH_INFO: `/v3/users/fa5d1b7938f8489c82b8420efdb34830` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:33.853950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5d1b7938f8489c82b8420efdb34830)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:33.856106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:33.910462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5d1b7938f8489c82b8420efdb34830'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.911177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5d1b7938f8489c82b8420efdb34830'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.942413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5d1b7938f8489c82b8420efdb34830'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.943644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5d1b7938f8489c82b8420efdb34830'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.956210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ee57ab4d-56b6-46c6-8138-b92684701f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5d1b7938f8489c82b8420efdb34830'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:33.976049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1217/2425] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:33 2022] DELETE /identity/v3/users/fa5d1b7938f8489c82b8420efdb34830 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:34.209924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:34.211386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:34.225313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49de463-287c-4a16-9f6a-3d5057d9af40 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:34.292184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:34.293475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:34.293758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:34.294033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] PATH_INFO: `/v3/projects/e7186bcef7054f14b10d8ae76e133857` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:34.295587 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7186bcef7054f14b10d8ae76e133857)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:34.297606 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:34.348535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7186bcef7054f14b10d8ae76e133857', 'request_id': 'req-ce0891fe-8e70-4041-bb26-65252da531fb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:34.358178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7186bcef7054f14b10d8ae76e133857', 'request_id': 'req-ce0891fe-8e70-4041-bb26-65252da531fb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:34.366478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7186bcef7054f14b10d8ae76e133857', 'request_id': 'req-ce0891fe-8e70-4041-bb26-65252da531fb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:34.386368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] The token cache is being invalidate because project e7186bcef7054f14b10d8ae76e133857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:34.386999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:34.388081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ce0891fe-8e70-4041-bb26-65252da531fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:34.390647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1209/2426] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:34 2022] DELETE /identity/v3/projects/e7186bcef7054f14b10d8ae76e133857 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:34.395816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:34.396484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:34.396781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:34.499422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:34.500820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:34.624447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1218/2427] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:34.626452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:34.626922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:34.627081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:34.689313 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:34.696728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:34.736113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1210/2428] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:34.736803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:34.764272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849f00c-64c4-454a-ad80-d93fcff59fa3 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:34.801738 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:34.802447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:34.802731 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:34.803005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:34.803758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:34.804063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:34.805716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:34.858656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1219/2429] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:35:34.871608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:34.887359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39848301-aa1d-4720-bbc0-f9cc62b38933 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:34.915899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:34.916889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:34.917270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:34.917983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:34.918916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:34.921527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:34.996573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:34.996823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1211/2430] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:34 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:35.019875 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb47f9-43c0-455d-b3a8-562f5c9eee7c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:35.041364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:35.258554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:35.259749 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:35.260254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:35.260917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:35.261865 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:35.264256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:35.311241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1220/2431] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:34 2022] POST /identity/v3/users => generated 292 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:35.315696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:35.330799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e01c2-4df6-4db8-961f-9d58866764ad admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:35.336229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:35.337388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:35.337940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:35.338463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:35.339494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:35.341577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:35.357841 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1212/2432] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:35.362517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:35.378219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb0f65-d821-4983-b767-55fc886758f0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:35.396176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad516c-0d71-45c5-b02c-baaa1a0ade33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:35.397429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad516c-0d71-45c5-b02c-baaa1a0ade33 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:35.397711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad516c-0d71-45c5-b02c-baaa1a0ade33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:35.397981 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad516c-0d71-45c5-b02c-baaa1a0ade33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:35.398733 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad516c-0d71-45c5-b02c-baaa1a0ade33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:35.400535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad516c-0d71-45c5-b02c-baaa1a0ade33 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:35.444430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1221/2433] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:35.445857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:35.467550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881a37-8e46-4da5-937d-3cbc6e2e5e37 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:35.476647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:35.478983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:35.479573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:35.480031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] PATH_INFO: `/v3/projects/b7fa93f5662947129a6e039bba99bfb3/users/ce643b7bb913434daa2046d7645f269a/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:35.509594 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7fa93f5662947129a6e039bba99bfb3, user_id=ce643b7bb913434daa2046d7645f269a, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:35.512872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:35.543759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:35.569267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1213/2434] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:35 2022] PUT /identity/v3/projects/b7fa93f5662947129a6e039bba99bfb3/users/ce643b7bb913434daa2046d7645f269a/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:35.570477 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:35.570773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:35.571069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:35.617712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] MFA Rules not processed for user `ce643b7bb913434daa2046d7645f269a`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:35.619102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:35.728863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1222/2435] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:35.733258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:35.952786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b89324-37a7-46d5-b3da-71fc1bc7ace0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:35.962950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:35.963723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:35.964005 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:35.975006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:35.976617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:35.979031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:36.053703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1214/2436] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:35 2022] POST /identity/v3/projects => generated 385 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:36.057492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:36.080499 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfacf4-dc3d-467f-960e-38ef4f690c0f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:36.252042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:36.252956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:36.270780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:36.271660 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:36.273544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:36.293580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:36.351439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1223/2437] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:36 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:36.355720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:36.373062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309e985-b6b0-4b38-a98f-f903b66b001a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:36.379390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:36.380397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:36.380950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:36.385295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:36.386580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:36.389112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:36.407458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1215/2438] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:36.411345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:36.431315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b935c-4e26-461e-9089-b634fd8a58a2 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:36.455797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffcd3c-69c9-41d4-9baf-e89727bac81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:36.458419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcd3c-69c9-41d4-9baf-e89727bac81e admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:36.458555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcd3c-69c9-41d4-9baf-e89727bac81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:36.459013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcd3c-69c9-41d4-9baf-e89727bac81e admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:36.460592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffcd3c-69c9-41d4-9baf-e89727bac81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:36.463399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffcd3c-69c9-41d4-9baf-e89727bac81e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:36.465262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:36.493381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1224/2439] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:36.493621 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:36.517093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a3040-7050-420c-840f-d5b0223485d6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:36.525909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:36.527015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:36.527300 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:36.527720 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] PATH_INFO: `/v3/projects/57df070449764560af6cfafc7801d787/users/6ba5be6f161349a6b51327949cfc6cb5/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:36.569749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57df070449764560af6cfafc7801d787, user_id=6ba5be6f161349a6b51327949cfc6cb5, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:36.573117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:36.594339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1216/2440] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:36 2022] PUT /identity/v3/projects/57df070449764560af6cfafc7801d787/users/6ba5be6f161349a6b51327949cfc6cb5/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:36.604540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:36.604907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:36.605347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:36.644214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] MFA Rules not processed for user `6ba5be6f161349a6b51327949cfc6cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:36.645834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:36.748295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1225/2441] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:35:36.753453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:36.862028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e988-fd7c-4c60-8ffa-13f2d0ef1364 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:36.877429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:36.878338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:36.878476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:36.878714 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:36.879474 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:36.883033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:36.929218 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1217/2442] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:36 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:36.935138 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:36.949050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f25a5-25e6-4fd2-998c-8e70ce060140 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:37.017337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:37.018021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.018303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.018718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.019411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:37.021397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:37.051973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1226/2443] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:36 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:37.056278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:37.070991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09184248-7ad0-4cb6-804a-5bd9d1a567a0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:37.082055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:37.082864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.083249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.083767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.084856 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:37.086585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:37.104897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1218/2444] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:35:37.110164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:37.120495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac728dab-308f-489e-94ea-e7f8e41c40e9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:37.128454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:37.129537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.129890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.132942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] PATH_INFO: `/v3/projects/e750d8d6b2274918ac2f0e6cc4dc511d/users/f9bcc27e26a0443aa54bfe730958c4a5/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.151913 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e750d8d6b2274918ac2f0e6cc4dc511d, user_id=f9bcc27e26a0443aa54bfe730958c4a5, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:37.155699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:37.193585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1227/2445] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:37 2022] PUT /identity/v3/projects/e750d8d6b2274918ac2f0e6cc4dc511d/users/f9bcc27e26a0443aa54bfe730958c4a5/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:37.197775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:37.208414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddbbf-c225-446f-ae40-601d59bc26c9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:37.229565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:37.284156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782b30-99a9-4300-bbb5-b9ffd937abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:37.285053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782b30-99a9-4300-bbb5-b9ffd937abf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.285437 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782b30-99a9-4300-bbb5-b9ffd937abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.286009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782b30-99a9-4300-bbb5-b9ffd937abf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.286883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782b30-99a9-4300-bbb5-b9ffd937abf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:37.288846 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782b30-99a9-4300-bbb5-b9ffd937abf6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:37.310764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1219/2446] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:37.311755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:37.327884 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f23d2-6ff0-4abe-a5ee-095b7eb455ba admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:37.336691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:37.337866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.338723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.343623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] PATH_INFO: `/v3/projects/e750d8d6b2274918ac2f0e6cc4dc511d/users/f9bcc27e26a0443aa54bfe730958c4a5/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.346324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e750d8d6b2274918ac2f0e6cc4dc511d, user_id=f9bcc27e26a0443aa54bfe730958c4a5, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:37.349588 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:37.380619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1228/2447] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:37 2022] PUT /identity/v3/projects/e750d8d6b2274918ac2f0e6cc4dc511d/users/f9bcc27e26a0443aa54bfe730958c4a5/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:37.393783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.394151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.394407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.451963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] MFA Rules not processed for user `f9bcc27e26a0443aa54bfe730958c4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:37.453489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:37.611498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1220/2448] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:37.625532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:37.627611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:37.709146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:37.749496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647eca4-43ad-4375-8419-4bf7b6489979 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:37.759096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:37.760033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:37.760677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:37.760941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:37.885241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:37.887117 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:37.907734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1229/2449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:39.445327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:39.448203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:39.560008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f20f5a-2b3e-425b-99ed-e3eedc447403 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:39.571762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:39.572512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:39.573271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:39.573626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:39.576014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:39.629785 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:39.633732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:39.641323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:39.646782 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c61fd-0e8b-42ac-8489-a99874f232d5 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:39.650927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:39.651752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:39.652043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:39.652311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:39.666257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:39.668102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:39.679252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1221/2450] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:39.712223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:39.713866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:39.723346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1230/2451] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:40.352306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:40.372583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1832a-fc0f-4477-977e-a88bf68ed8c9 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:40.376938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16095c65-7a90-4aa4-b66d-d238bff89a81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:40.377686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16095c65-7a90-4aa4-b66d-d238bff89a81 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:40.377973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16095c65-7a90-4aa4-b66d-d238bff89a81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:40.378323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16095c65-7a90-4aa4-b66d-d238bff89a81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:40.389397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16095c65-7a90-4aa4-b66d-d238bff89a81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:40.390892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16095c65-7a90-4aa4-b66d-d238bff89a81 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:40.401287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1222/2452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:40.673167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:40.675846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:40.764107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbca7b9-a1b8-474f-ab64-1bcc60da710a service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:40.769156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:40.770200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:40.770573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:40.770919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] PATH_INFO: `/v3/users/3abffceec7bb45eba8ae593903d6ac85` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:40.772214 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3abffceec7bb45eba8ae593903d6ac85)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:40.774472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:40.820601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:40.838121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3abffceec7bb45eba8ae593903d6ac85'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:40.838989 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3abffceec7bb45eba8ae593903d6ac85'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:40.856026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3abffceec7bb45eba8ae593903d6ac85'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:40.857702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3abffceec7bb45eba8ae593903d6ac85'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:40.868791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3abffceec7bb45eba8ae593903d6ac85'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:40.892260 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1231/2453] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:40 2022] DELETE /identity/v3/users/3abffceec7bb45eba8ae593903d6ac85 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:40.896907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:40.897420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:40.897524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:40.914890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:40.916112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:40.988135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1223/2454] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:40.997628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:40.999847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:41.064930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046add7f-061c-4223-9b23-e75ad352a026 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:41.068582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a1161-aaa8-4f31-b7a2-324e24b28623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:41.069264 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a1161-aaa8-4f31-b7a2-324e24b28623 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:41.069544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a1161-aaa8-4f31-b7a2-324e24b28623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:41.069818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a1161-aaa8-4f31-b7a2-324e24b28623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:41.079308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a1161-aaa8-4f31-b7a2-324e24b28623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:41.080802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a1161-aaa8-4f31-b7a2-324e24b28623 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:41.090835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1232/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:41.519383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:41.521538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:41.547062 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52abf6-3dd2-48a8-8561-e87684e31308 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:41.559716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:41.561595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:41.561942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:41.562231 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] PATH_INFO: `/v3/projects/3bd7e4eb5c9240388d7e8744afdecb3d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:41.563515 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd7e4eb5c9240388d7e8744afdecb3d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:41.566055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:41.657331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd7e4eb5c9240388d7e8744afdecb3d', 'request_id': 'req-09185e93-13a4-46d3-a219-f51fb72d1153'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:41.671671 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd7e4eb5c9240388d7e8744afdecb3d', 'request_id': 'req-09185e93-13a4-46d3-a219-f51fb72d1153'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:41.679466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd7e4eb5c9240388d7e8744afdecb3d', 'request_id': 'req-09185e93-13a4-46d3-a219-f51fb72d1153'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:41.695337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] The token cache is being invalidate because project 3bd7e4eb5c9240388d7e8744afdecb3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:41.695623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:41.696306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-09185e93-13a4-46d3-a219-f51fb72d1153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:41.706145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1224/2456] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:41 2022] DELETE /identity/v3/projects/3bd7e4eb5c9240388d7e8744afdecb3d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:41.706653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:41.706653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:41.706653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:41.733976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:41.735173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:41.845681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1233/2457] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:41.854534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:41.855586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:41.856066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:41.890550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:41.892456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:41.905077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1225/2458] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:41.909209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:41.932315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5116ec-dd62-47be-92a4-acc17e040e37 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:41.939557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:41.941431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:41.941926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:41.942741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:41.943778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:41.944803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:41.946730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:41.971412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1234/2459] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:35:41.975920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:41.989536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d547c-68f6-4b17-a70a-da1aa95cd661 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:41.995394 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:41.997935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:41.999188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:41.999794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.000897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:42.008305 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:42.058140 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1226/2460] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:41 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:42.062421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:42.078306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ff491-fe50-45e0-aff6-5165516bc910 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:42.128409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:42.157017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:42.157715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:42.158047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:42.158329 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.159024 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:42.161178 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:42.202294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1235/2461] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:42 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:42.206504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:42.222756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171309-b6a0-408b-a353-1c22cb6ee384 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:42.228042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:42.228842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:42.229726 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:42.230222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.231180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:42.233747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:42.253657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1227/2462] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:42.255727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:42.271218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66117ada-19cb-4f29-9948-0376acb97d5d admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:42.278123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7817456-cf86-444c-b466-8a004312e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:42.278825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7817456-cf86-444c-b466-8a004312e6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:42.279158 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7817456-cf86-444c-b466-8a004312e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:42.279519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7817456-cf86-444c-b466-8a004312e6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.280378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7817456-cf86-444c-b466-8a004312e6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:42.282522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7817456-cf86-444c-b466-8a004312e6a5 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:42.295735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1236/2463] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:42.300186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:42.313624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14a42d-16d8-4a85-844f-411df772daf2 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:42.320149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:42.321128 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:42.321458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:42.321736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] PATH_INFO: `/v3/projects/03bd1ea72ee241f3b0d311c0dd5f439a/users/09c9aadd8f87449bae9a725e511e13be/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.324780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:42.349050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03bd1ea72ee241f3b0d311c0dd5f439a, user_id=09c9aadd8f87449bae9a725e511e13be, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:42.351477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:42.370913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1228/2464] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:42 2022] PUT /identity/v3/projects/03bd1ea72ee241f3b0d311c0dd5f439a/users/09c9aadd8f87449bae9a725e511e13be/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:42.450012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:42.451432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:42.452199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.523537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] MFA Rules not processed for user `09c9aadd8f87449bae9a725e511e13be`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:42.524697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:42.659146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1237/2465] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:42.671554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:42.673668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:42.796882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c7afc-bb2c-49fb-84da-2856b8ae12c0 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:42.801511 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4c5bf-0dd6-4162-bfdf-2e19e4018510 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:42.802359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4c5bf-0dd6-4162-bfdf-2e19e4018510 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:42.802688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4c5bf-0dd6-4162-bfdf-2e19e4018510 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:42.802924 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4c5bf-0dd6-4162-bfdf-2e19e4018510 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:42.819254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4c5bf-0dd6-4162-bfdf-2e19e4018510 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:42.828558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4c5bf-0dd6-4162-bfdf-2e19e4018510 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:42.839435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1229/2466] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:44.786992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:44.788458 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:44.901510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e000bf-b627-4315-ad0a-9f7a3c1c10e5 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:44.908995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:44.910031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:44.910799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:44.911199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] PATH_INFO: `/v3/users/ce643b7bb913434daa2046d7645f269a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:44.913144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce643b7bb913434daa2046d7645f269a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:44.916241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:44.971666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce643b7bb913434daa2046d7645f269a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:44.972365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce643b7bb913434daa2046d7645f269a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:44.974591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:44.989171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce643b7bb913434daa2046d7645f269a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:44.990240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce643b7bb913434daa2046d7645f269a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.000203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce643b7bb913434daa2046d7645f269a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.014118 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1238/2467] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:44 2022] DELETE /identity/v3/users/ce643b7bb913434daa2046d7645f269a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:45.018870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:45.019460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:45.019876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:45.053510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:45.058291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:45.150280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1230/2468] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:35:45.159924 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:45.161367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:45.231796 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a08892-3461-4db7-8b63-6cd9396d4963 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:45.235754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:45.236416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:45.236727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:45.237007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:45.246209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:45.247692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:45.259617 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1239/2469] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:45.586947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:45.589064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:45.614257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5800d4-6112-4727-bbdb-125f841eaecf None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:45.621413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:45.622470 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:45.622795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:45.623069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] PATH_INFO: `/v3/projects/b7fa93f5662947129a6e039bba99bfb3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:45.624386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7fa93f5662947129a6e039bba99bfb3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:45.626430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:45.696980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7fa93f5662947129a6e039bba99bfb3', 'request_id': 'req-fced8985-c5c2-4dc8-b506-c36bb9b861c7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.709168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7fa93f5662947129a6e039bba99bfb3', 'request_id': 'req-fced8985-c5c2-4dc8-b506-c36bb9b861c7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.715731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7fa93f5662947129a6e039bba99bfb3', 'request_id': 'req-fced8985-c5c2-4dc8-b506-c36bb9b861c7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.730708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] The token cache is being invalidate because project b7fa93f5662947129a6e039bba99bfb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:45.731021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.731643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.737790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1231/2470] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:45 2022] DELETE /identity/v3/projects/b7fa93f5662947129a6e039bba99bfb3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:45.742261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:45.754529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362c2a-3f59-420c-94bb-f4a487d862ef service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:45.826831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:45.827675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:45.827991 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:45.828277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] PATH_INFO: `/v3/users/6ba5be6f161349a6b51327949cfc6cb5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:45.829663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba5be6f161349a6b51327949cfc6cb5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:45.831798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:45.907001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba5be6f161349a6b51327949cfc6cb5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.908029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba5be6f161349a6b51327949cfc6cb5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.928047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba5be6f161349a6b51327949cfc6cb5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.928963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba5be6f161349a6b51327949cfc6cb5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.939326 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba5be6f161349a6b51327949cfc6cb5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:45.941720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:45.959793 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1240/2471] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:45 2022] DELETE /identity/v3/users/6ba5be6f161349a6b51327949cfc6cb5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:46.543473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:46.545715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:46.565757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8985-c5c2-4dc8-b506-c36bb9b861c7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:46.571771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:46.638643 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:46.639537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:46.639819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:46.640104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] PATH_INFO: `/v3/projects/57df070449764560af6cfafc7801d787` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:46.641452 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57df070449764560af6cfafc7801d787)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:46.643502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:46.693738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57df070449764560af6cfafc7801d787', 'request_id': 'req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.701921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57df070449764560af6cfafc7801d787', 'request_id': 'req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.708412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57df070449764560af6cfafc7801d787', 'request_id': 'req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.720999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] The token cache is being invalidate because project 57df070449764560af6cfafc7801d787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:46.721286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.722007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.723674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1232/2472] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:46 2022] DELETE /identity/v3/projects/57df070449764560af6cfafc7801d787 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:46.727797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:46.729250 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:46.743089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9140b06-46db-4979-880f-7b888cb5f322 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:46.822397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:46.823288 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:46.823567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:46.823837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] PATH_INFO: `/v3/users/f9bcc27e26a0443aa54bfe730958c4a5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:46.825058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9bcc27e26a0443aa54bfe730958c4a5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:46.827376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:46.904275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9bcc27e26a0443aa54bfe730958c4a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.904965 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9bcc27e26a0443aa54bfe730958c4a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.929706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9bcc27e26a0443aa54bfe730958c4a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.930847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9bcc27e26a0443aa54bfe730958c4a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.946787 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e93ade75-6bed-4dac-8e9a-80737e5db22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9bcc27e26a0443aa54bfe730958c4a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:46.965744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1241/2473] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:35:46 2022] DELETE /identity/v3/users/f9bcc27e26a0443aa54bfe730958c4a5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:47.329843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:47.331933 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:47.345676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80131d-3c95-4848-ba6a-0b2c3c1dbec6 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:47.441556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:47.442338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.442623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.442918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] PATH_INFO: `/v3/projects/e750d8d6b2274918ac2f0e6cc4dc511d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.447332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e750d8d6b2274918ac2f0e6cc4dc511d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:47.449626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:47.502048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:47.508698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e750d8d6b2274918ac2f0e6cc4dc511d', 'request_id': 'req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:47.520898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e750d8d6b2274918ac2f0e6cc4dc511d', 'request_id': 'req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:47.531865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e750d8d6b2274918ac2f0e6cc4dc511d', 'request_id': 'req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:47.549290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] The token cache is being invalidate because project e750d8d6b2274918ac2f0e6cc4dc511d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:35:47.550057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:47.550228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b9cefb67-e2e2-4bbd-8ba5-b63b9ea70f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:35:47.557103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1233/2474] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:35:47 2022] DELETE /identity/v3/projects/e750d8d6b2274918ac2f0e6cc4dc511d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:35:47.558993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.559277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.559573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.579369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:47.580590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:47.649519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1242/2475] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:47.657668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.657895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.658444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.677529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:47.678740 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:47.690332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1234/2476] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:47.694721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:47.705818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9775aa5-d391-4720-b32d-b81ae5a8a314 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:47.715253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:47.716348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.716767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.717162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.718040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:35:47.718765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:47.720515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:47.723573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:35:47.736672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1243/2477] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:35:47.740752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:47.750458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd86c1-c325-4ea3-8c74-0f038faef670 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:47.755375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:47.756089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.756410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.756553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.757721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:47.759191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:47.795478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1235/2478] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:35:47 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:47.796856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:47.807564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277fa1-df3b-44c9-ba45-c5f9a8d46b88 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:47.874305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:47.875000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.875247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.875515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.876208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:47.878281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:47.904916 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1244/2479] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:35:47 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:35:47.907654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:47.919700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3d933-d5e7-4a69-ac4a-5added70485a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:47.925742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:47.926435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.926724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.927006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.927746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:47.929493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:47.941616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1236/2480] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:35:47.945377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:47.955939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a839-1fb0-4716-89d5-6145e381b238 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:47.961080 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0eb3b-32cc-4880-85c5-7b0642400c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:47.961709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0eb3b-32cc-4880-85c5-7b0642400c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:47.962001 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0eb3b-32cc-4880-85c5-7b0642400c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:47.962279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0eb3b-32cc-4880-85c5-7b0642400c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:47.963022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0eb3b-32cc-4880-85c5-7b0642400c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:47.964708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0eb3b-32cc-4880-85c5-7b0642400c5f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:47.976932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1245/2481] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:35:47.981287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:48.003535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c77d4d-7838-4d92-a255-843c5a1b1fa8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:48.008363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:48.009369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:48.009751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:48.010089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] PATH_INFO: `/v3/projects/d64b4dd176f64980bf01a4f17b221e66/users/fe79ea5a26444dc4b36986b3ad7c9236/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:48.032454 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64b4dd176f64980bf01a4f17b221e66, user_id=fe79ea5a26444dc4b36986b3ad7c9236, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:48.035700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:48.053982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1237/2482] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:35:47 2022] PUT /identity/v3/projects/d64b4dd176f64980bf01a4f17b221e66/users/fe79ea5a26444dc4b36986b3ad7c9236/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:35:48.066421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de58ee-fbe0-4f4d-8c41-64b627bcabf5 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:48.067003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de58ee-fbe0-4f4d-8c41-64b627bcabf5 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:48.067320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de58ee-fbe0-4f4d-8c41-64b627bcabf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:48.098955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-04de58ee-fbe0-4f4d-8c41-64b627bcabf5 None None] MFA Rules not processed for user `fe79ea5a26444dc4b36986b3ad7c9236`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:35:48.100483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-04de58ee-fbe0-4f4d-8c41-64b627bcabf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:48.176231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1246/2483] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:35:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:35:55.463032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:35:55.465335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:35:55.537115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddfcbe-35da-4497-90fa-99fd17d0266f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:35:55.540811 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:35:55.541463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:35:55.542102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:35:55.542420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:35:55.551219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:35:55.552747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:35:55.562320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1238/2484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:35:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:35:58.174279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:02.895855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:02.896603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:02.896603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:02.915904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] MFA Rules not processed for user `72419918d6c34080b6a4a4c73b2c06b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:02.917699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:02.978892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1247/2485] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:04.273425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:04.275018 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:04.352494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:04.379531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d9ab8-431e-4393-96c3-b75f15bfba50 service glance] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:04.385451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:04.386192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:04.386475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:04.386747 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:04.388717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:04.401200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:04.402792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:04.413973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1239/2486] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:12.971003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=25 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:14.409156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=10 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:24.022960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:24.024614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:24.088218 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8721f-082e-4c8b-b6d5-c1fd7b2d8065 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:24.092237 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:24.092935 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:24.093208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:24.093531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:24.095648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:24.162795 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:24.165161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:24.176135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1248/2487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:26.246997 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:26.266236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5738a-cd5e-49d4-9869-7f6a6e689b19 service placement] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:26.272453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:26.274631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:26.274977 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:26.275705 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:26.287089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:26.434918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:26.436645 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00817a03-dbaf-40d9-bae6-9cbf512e1cf9 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:26.455058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1240/2488] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:33.539109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:33.541619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:33.611136 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affea872-b146-4d7f-ac1d-ef16db7ed4bb service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:33.615906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:33.616820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:33.617095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:33.617358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] PATH_INFO: `/v3/users/09c9aadd8f87449bae9a725e511e13be` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:33.618572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c9aadd8f87449bae9a725e511e13be)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:33.620489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:33.670866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c9aadd8f87449bae9a725e511e13be'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:33.671518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c9aadd8f87449bae9a725e511e13be'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:33.676606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:33.686510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c9aadd8f87449bae9a725e511e13be'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:33.687205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c9aadd8f87449bae9a725e511e13be'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:33.695556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c9aadd8f87449bae9a725e511e13be'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:33.708181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1249/2489] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:36:33 2022] DELETE /identity/v3/users/09c9aadd8f87449bae9a725e511e13be => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:33.712727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:33.713228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:33.713340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:33.730227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:33.731660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:33.796227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1241/2490] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:33.805929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:33.807475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:33.866867 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b732-4b17-402f-887c-729110c03deb admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:33.871154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:33.871841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:33.872123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:33.872394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:33.882404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:33.883852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:33.894813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1250/2491] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:36:34.170753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:34.172340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:34.184840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc5ddd-40a2-4e65-91f1-82e6858d6637 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:34.190691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:34.191692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:34.191976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:34.192290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] PATH_INFO: `/v3/projects/03bd1ea72ee241f3b0d311c0dd5f439a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:34.195694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03bd1ea72ee241f3b0d311c0dd5f439a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:34.197695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:34.253914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03bd1ea72ee241f3b0d311c0dd5f439a', 'request_id': 'req-0be78bda-c494-400e-a5fb-df62d968831a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:34.263585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03bd1ea72ee241f3b0d311c0dd5f439a', 'request_id': 'req-0be78bda-c494-400e-a5fb-df62d968831a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:34.273463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03bd1ea72ee241f3b0d311c0dd5f439a', 'request_id': 'req-0be78bda-c494-400e-a5fb-df62d968831a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:34.286302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] The token cache is being invalidate because project 03bd1ea72ee241f3b0d311c0dd5f439a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:36:34.286635 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:34.287373 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-0be78bda-c494-400e-a5fb-df62d968831a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:34.289069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1242/2492] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:36:34 2022] DELETE /identity/v3/projects/03bd1ea72ee241f3b0d311c0dd5f439a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:35.238284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:35.239977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:35.255156 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c26259-256d-4441-b1d1-da38b2b0ab43 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:35.326186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:35.327075 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:35.327358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:35.327632 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] PATH_INFO: `/v3/users/fe79ea5a26444dc4b36986b3ad7c9236` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:35.328968 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe79ea5a26444dc4b36986b3ad7c9236)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:35.330911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:35.385680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe79ea5a26444dc4b36986b3ad7c9236'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:35.386475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe79ea5a26444dc4b36986b3ad7c9236'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:35.398611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe79ea5a26444dc4b36986b3ad7c9236'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:35.399303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe79ea5a26444dc4b36986b3ad7c9236'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:35.406979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe79ea5a26444dc4b36986b3ad7c9236'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:35.409134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:35.422298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1251/2493] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:36:35 2022] DELETE /identity/v3/users/fe79ea5a26444dc4b36986b3ad7c9236 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:35.426963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:35.427442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:35.427442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:35.446459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:35.447684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:35.463154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:35.515859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1243/2494] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:35.525735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:35.527315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:35.604443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078c4c-aaeb-44ab-b7f6-c8f0e86f910b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:35.609385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5c678-489a-4a85-ba0e-988dc1f195e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:35.610070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5c678-489a-4a85-ba0e-988dc1f195e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:35.610348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5c678-489a-4a85-ba0e-988dc1f195e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:35.610618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5c678-489a-4a85-ba0e-988dc1f195e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:35.621162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5c678-489a-4a85-ba0e-988dc1f195e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:35.622716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5c678-489a-4a85-ba0e-988dc1f195e4 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:35.633003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1252/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:35.925127 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:35.926576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:35.938902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926df1-97a0-480c-8324-d69eb52694a2 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:35.944079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:35.945054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:35.945335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:35.945605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] PATH_INFO: `/v3/projects/d64b4dd176f64980bf01a4f17b221e66` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:35.946843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64b4dd176f64980bf01a4f17b221e66)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:35.948815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.002165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64b4dd176f64980bf01a4f17b221e66', 'request_id': 'req-97180d2f-f763-4d0b-b76c-13c32d11475d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.011038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64b4dd176f64980bf01a4f17b221e66', 'request_id': 'req-97180d2f-f763-4d0b-b76c-13c32d11475d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.019332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64b4dd176f64980bf01a4f17b221e66', 'request_id': 'req-97180d2f-f763-4d0b-b76c-13c32d11475d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.032838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] The token cache is being invalidate because project d64b4dd176f64980bf01a4f17b221e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:36:36.033275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.034113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-97180d2f-f763-4d0b-b76c-13c32d11475d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.035976 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1244/2496] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:36:35 2022] DELETE /identity/v3/projects/d64b4dd176f64980bf01a4f17b221e66 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:36.041445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.041734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.042011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.064736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:36.065977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:36.129680 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1253/2497] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:36.137565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.137832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.138154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.157204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:36.158407 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:36.167054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1245/2498] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:36.171531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.181446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fe4dc-f7fc-451b-be02-b1fa974b3b95 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.187905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.188993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.189365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.189760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.190941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:36:36.191400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.194079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.207977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1254/2499] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:36:36.212292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.221589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619793b7-4c57-4a68-b288-0e44db8b91b6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.226519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.227173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.227456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.227759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.228453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.230439 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.260248 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1246/2500] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:36:36 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:36:36.264656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.274879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48f3b4-19bb-46bf-afb1-3c2756df0711 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.333858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.334529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.334808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.335079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.335762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.337747 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.361270 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1255/2501] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:36:36 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:36:36.366147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.380383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d1a44-f92b-48fe-a7a9-8d8c34cb0670 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.386495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.387553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.387553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.387795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.388529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.390239 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.401672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1247/2502] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:36:36.405898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.417752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff36b-e665-43d6-a9f3-87ea04f43c23 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.423160 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7566c1-d64e-4fa9-83ac-5111361d4624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.423837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7566c1-d64e-4fa9-83ac-5111361d4624 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.424141 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7566c1-d64e-4fa9-83ac-5111361d4624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.424415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7566c1-d64e-4fa9-83ac-5111361d4624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.425202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7566c1-d64e-4fa9-83ac-5111361d4624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.426828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7566c1-d64e-4fa9-83ac-5111361d4624 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.432413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:36.438872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1256/2503] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:36:36.444233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.455052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e49a7-616e-4dc7-9686-ea3233d7671a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.460014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.460937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.461213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.461483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] PATH_INFO: `/v3/projects/f9a5799854d34d4fae8e02fe2ad0fbe6/users/fc2c1a7a63b043efb9adbba7706f60e4/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.476892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a5799854d34d4fae8e02fe2ad0fbe6, user_id=fc2c1a7a63b043efb9adbba7706f60e4, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.479559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.495161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1248/2504] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:36:36 2022] PUT /identity/v3/projects/f9a5799854d34d4fae8e02fe2ad0fbe6/users/fc2c1a7a63b043efb9adbba7706f60e4/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:36.506926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.507283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.507619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.537376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] MFA Rules not processed for user `fc2c1a7a63b043efb9adbba7706f60e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:36.538595 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:36.614958 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1257/2505] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:36.626806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.628517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:36.683318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:36.745698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860544a-e8ed-4f40-b0c7-d5d11ad5b80a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.751915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12720f45-3eae-4fa8-8368-17dc1bdfefe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.752685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12720f45-3eae-4fa8-8368-17dc1bdfefe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.752973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12720f45-3eae-4fa8-8368-17dc1bdfefe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.753250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12720f45-3eae-4fa8-8368-17dc1bdfefe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.765182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12720f45-3eae-4fa8-8368-17dc1bdfefe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.767429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12720f45-3eae-4fa8-8368-17dc1bdfefe0 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.779950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1249/2506] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:36.804898 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:36.870391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ef515-a183-4c6d-8b7e-955bd02bd3b9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:36.875693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:36.876565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.876877 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.877151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] PATH_INFO: `/v3/users/fc2c1a7a63b043efb9adbba7706f60e4` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.878389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2c1a7a63b043efb9adbba7706f60e4)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:36.880322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:36.934739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2c1a7a63b043efb9adbba7706f60e4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.935486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2c1a7a63b043efb9adbba7706f60e4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.948874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2c1a7a63b043efb9adbba7706f60e4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.949684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2c1a7a63b043efb9adbba7706f60e4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.958559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2c1a7a63b043efb9adbba7706f60e4'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:36.970861 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1258/2507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:36:36 2022] DELETE /identity/v3/users/fc2c1a7a63b043efb9adbba7706f60e4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:36.975440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:36.975733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:36.976012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:36.993788 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:36.995028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:37.059943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1250/2508] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:37.072029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.073670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:37.086404 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:37.135379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053dcbd-fd52-4742-b4c1-883fc8d803f6 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.139887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac8f60-6daf-4437-85a8-5b5e85d0d708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.140561 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac8f60-6daf-4437-85a8-5b5e85d0d708 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.140876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac8f60-6daf-4437-85a8-5b5e85d0d708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.141152 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac8f60-6daf-4437-85a8-5b5e85d0d708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.150639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac8f60-6daf-4437-85a8-5b5e85d0d708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.152091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac8f60-6daf-4437-85a8-5b5e85d0d708 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.163035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1259/2509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:37.459909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.462159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:37.475929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814f75e-a292-4bfa-a515-9dbafffe3262 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.481098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.482448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.482735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.483010 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] PATH_INFO: `/v3/projects/f9a5799854d34d4fae8e02fe2ad0fbe6` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.484235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a5799854d34d4fae8e02fe2ad0fbe6)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.486208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.552355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a5799854d34d4fae8e02fe2ad0fbe6', 'request_id': 'req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:37.561271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a5799854d34d4fae8e02fe2ad0fbe6', 'request_id': 'req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:37.566920 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a5799854d34d4fae8e02fe2ad0fbe6', 'request_id': 'req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:37.578221 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] The token cache is being invalidate because project f9a5799854d34d4fae8e02fe2ad0fbe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:36:37.578505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:37.579146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b40b8067-71e8-4534-bf6d-7a6f83fce0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:36:37.580842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1251/2510] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:36:37 2022] DELETE /identity/v3/projects/f9a5799854d34d4fae8e02fe2ad0fbe6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:36:37.585967 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.586447 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.586597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.604567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:37.606220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:37.677547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1260/2511] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:37.686560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.686876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.687155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.706681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:37.707898 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:37.717729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1252/2512] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:37.722168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.732484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547cd4c-6174-4efe-86bb-e361562acb0c None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.737801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.738465 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.738750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.739058 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.739786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:36:37.740093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.741721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.753744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1261/2513] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:36:37.758081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.767828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352e5b-ae14-4051-889f-ea8717b044ea None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.773823 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.774471 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.774753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.775020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.775701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.777678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.807207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1253/2514] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:36:37 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:36:37.811166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.821631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7d0e9-0bc5-46e6-8ae5-c78996191f43 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.886545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.887361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.887818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.887818 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.888536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.890503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.911956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1262/2515] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:36:37 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:36:37.916440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.927538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633c57-a5b0-46ac-9085-2cf0c058c905 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.933268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.934117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.934617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.934617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.935381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.937023 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.949262 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1254/2516] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:36:37.955271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:37.969656 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cb45-e2fa-4373-84ca-b6650a191cb7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:37.977144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8288e-cbd6-4978-a8e4-4fc1a6cebece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:37.978093 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8288e-cbd6-4978-a8e4-4fc1a6cebece admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:37.978449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8288e-cbd6-4978-a8e4-4fc1a6cebece admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:37.978842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8288e-cbd6-4978-a8e4-4fc1a6cebece admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:37.979969 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8288e-cbd6-4978-a8e4-4fc1a6cebece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:37.981999 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8288e-cbd6-4978-a8e4-4fc1a6cebece admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:37.992037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1263/2517] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:36:37.996233 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:38.002981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:38.008950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05bd7b-5f67-4803-9fe2-cfd913dc1056 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:38.016135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:38.017282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:38.017737 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:38.018033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] PATH_INFO: `/v3/projects/e3eabfe3692b44d8ae5f7d137ce1d157/users/4bdc0eeb6cd443d78aefefca488d8c5d/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:38.033377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3eabfe3692b44d8ae5f7d137ce1d157, user_id=4bdc0eeb6cd443d78aefefca488d8c5d, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:38.036040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:38.051089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1255/2518] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:36:37 2022] PUT /identity/v3/projects/e3eabfe3692b44d8ae5f7d137ce1d157/users/4bdc0eeb6cd443d78aefefca488d8c5d/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:36:38.057429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba8e83-4427-47c3-b5c0-ef2ff6464d71 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:38.057681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba8e83-4427-47c3-b5c0-ef2ff6464d71 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:38.058196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba8e83-4427-47c3-b5c0-ef2ff6464d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:38.077637 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-38ba8e83-4427-47c3-b5c0-ef2ff6464d71 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:38.078862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-38ba8e83-4427-47c3-b5c0-ef2ff6464d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:38.091516 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:38.142145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1264/2519] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:38.152139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:38.154139 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:38.225463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254a0fb-5d12-4b45-884e-e21be6a372ed admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:38.231828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:38.232509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:38.232814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:38.233118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:38.244113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:38.245566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:38.256475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1256/2520] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:38.916415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8eb01-48ee-4223-8cea-1f5290e4bb28 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:38.917005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8eb01-48ee-4223-8cea-1f5290e4bb28 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:38.917005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8eb01-48ee-4223-8cea-1f5290e4bb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:38.945338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-1fd8eb01-48ee-4223-8cea-1f5290e4bb28 None None] MFA Rules not processed for user `4bdc0eeb6cd443d78aefefca488d8c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:38.946557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1fd8eb01-48ee-4223-8cea-1f5290e4bb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:39.017879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1265/2521] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:39.028873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:36:39.030484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:39.093602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1bcab-1170-4eb7-bffc-0d8c70c728a1 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:36:39.097390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ff717-6130-4776-81e0-07dadc2cd008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:36:39.098048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ff717-6130-4776-81e0-07dadc2cd008 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:39.098327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ff717-6130-4776-81e0-07dadc2cd008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:39.098600 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ff717-6130-4776-81e0-07dadc2cd008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:39.107193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ff717-6130-4776-81e0-07dadc2cd008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:36:39.108618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ff717-6130-4776-81e0-07dadc2cd008 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:36:39.117301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1257/2522] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:36:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:36:40.523589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:36:40.524121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:36:40.524121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:36:40.543439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] MFA Rules not processed for user `4bdc0eeb6cd443d78aefefca488d8c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:36:40.544723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:36:40.553032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1266/2523] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:36:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:36:49.113909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:36:50.537375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=54 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:37:18.312283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf7a2-35eb-4d79-92d3-ed1052b340f6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:37:18.312881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf7a2-35eb-4d79-92d3-ed1052b340f6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:37:18.312881 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf7a2-35eb-4d79-92d3-ed1052b340f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:37:18.332412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ca1bf7a2-35eb-4d79-92d3-ed1052b340f6 None None] MFA Rules not processed for user `26807e5e7aab425eb28e781289794265`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:37:18.333902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ca1bf7a2-35eb-4d79-92d3-ed1052b340f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:18.395416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1258/2524] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:37:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:37:28.388436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=25 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:37:52.653953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:37:52.655371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:52.670382 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa429e-4ac1-4fb3-b579-5710bdf0d570 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:37:52.675758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:37:52.676414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:37:52.676727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:37:52.677002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:37:52.678973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:52.770676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:37:52.772047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:37:52.793743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1267/2525] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:37:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:37:55.742629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:37:55.743131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:37:55.743230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:37:55.762854 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] MFA Rules not processed for user `4bdc0eeb6cd443d78aefefca488d8c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:37:55.764056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:55.776993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1259/2526] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:37:55.808424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:37:55.810430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:55.901757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aafda-0a39-4157-930c-2040c37f7a40 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:37:55.905911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:37:55.906600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:37:55.907380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:37:55.907380 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:37:55.909267 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:55.924562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:37:55.926087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:37:55.938926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1268/2527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:37:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:37:58.375556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:37:58.389933 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63b2c-ec26-4069-8887-f02307d8f29b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:37:58.395003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:37:58.395677 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:37:58.395935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:37:58.396200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:37:58.398363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:58.402115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:37:58.403789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:58.427510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae092d-a098-4ef6-b124-09a53892a163 service glance] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:37:58.434356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:37:58.435101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:37:58.435950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:37:58.436125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:37:58.438396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:37:58.474184 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:37:58.476215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4628-a0d9-4bbe-b256-15776a030a5f service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:37:58.487091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1260/2528] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:37:58.489409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:37:58.508552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:37:58.510188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:37:58.520618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1269/2529] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:05.474564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1276f7-6b35-4a37-a5d0-6d3d7d6ef0b0 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:05.475088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1276f7-6b35-4a37-a5d0-6d3d7d6ef0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:05.475197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1276f7-6b35-4a37-a5d0-6d3d7d6ef0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:05.539889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3f1276f7-6b35-4a37-a5d0-6d3d7d6ef0b0 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:05.542036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3f1276f7-6b35-4a37-a5d0-6d3d7d6ef0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:05.610915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1261/2530] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jun 30 12:38:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:38:05.654663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:05.656287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:05.759902 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0366e55-00d7-4d27-be1e-a3d24b4d3402 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:05.765168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:05.765868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:05.766198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:05.766469 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:05.775379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:05.777030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:05.788102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1270/2531] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:38:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:38:06.405257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:06.405631 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:06.405996 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:06.425314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] MFA Rules not processed for user `6ddb38b4d6864a6c96b84c9a4092eea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:06.426546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:06.433797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1262/2532] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:38:07.204930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:07.215254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be12c2-b827-4469-9d1d-31cbb311adfe service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:07.219758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:07.220570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:07.220963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:07.221335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:07.223351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:07.233569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:07.235261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:07.245079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1271/2533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:07.776949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:07.779530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:07.864985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:07.887956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed6efa-bfd7-481a-9741-6f2feb261d5c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:07.894075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:07.894774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:07.895039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:07.896701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:07.898808 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:07.913064 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:07.914813 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:07.928686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1263/2534] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:10.665525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:10.679760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b591f9-7c48-404b-97f3-f4bbde014c47 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:10.683652 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:10.684449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:10.684872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:10.685390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:10.687569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:10.700729 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:10.702800 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:10.717185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1272/2535] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:15.598260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:15.612084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa688-1823-4e93-bb56-b8c921ec4a6e service placement] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:15.618222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:15.618895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:15.619179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:15.619601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:15.621814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:15.635356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:15.637188 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:15.651225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1264/2536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:15.690358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:15.693552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:15.714405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4177a-9278-4446-b57d-0700e20fe32d service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:15.722124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:15.723272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:15.723704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:15.723864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] PATH_INFO: `/v3/users/1fc65fea19f94fd18d83ad8ce3bbb476` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:15.725259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc65fea19f94fd18d83ad8ce3bbb476)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:15.727318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:15.779198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc65fea19f94fd18d83ad8ce3bbb476'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:15.779777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc65fea19f94fd18d83ad8ce3bbb476'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:15.797842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc65fea19f94fd18d83ad8ce3bbb476'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:15.799108 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc65fea19f94fd18d83ad8ce3bbb476'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:15.810354 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc65fea19f94fd18d83ad8ce3bbb476'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:15.817766 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:15.827705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1273/2537] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:38:15 2022] DELETE /identity/v3/users/1fc65fea19f94fd18d83ad8ce3bbb476 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:38:16.019538 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:16.043774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b46c49-cb1b-4844-81a7-ab0b4c7dae05 service placement] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:16.113634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:16.114508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:16.114784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:16.115045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] PATH_INFO: `/v3/projects/4a42d5073b0f4e4c857d794aa8d30393` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:16.116372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a42d5073b0f4e4c857d794aa8d30393)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:16.118355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:16.167199 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a42d5073b0f4e4c857d794aa8d30393', 'request_id': 'req-fc04dfd5-385b-45f9-a747-55cabb7801bc'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.176803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a42d5073b0f4e4c857d794aa8d30393', 'request_id': 'req-fc04dfd5-385b-45f9-a747-55cabb7801bc'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.182412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a42d5073b0f4e4c857d794aa8d30393', 'request_id': 'req-fc04dfd5-385b-45f9-a747-55cabb7801bc'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.193718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] The token cache is being invalidate because project 4a42d5073b0f4e4c857d794aa8d30393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:38:16.194004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.194648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.196261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1265/2538] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:38:16 2022] DELETE /identity/v3/projects/4a42d5073b0f4e4c857d794aa8d30393 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:38:16.200364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:16.201856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:16.215478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436e26d-202b-4e7c-8595-a1d97b0b3485 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:16.273343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:16.274227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:16.274505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:16.274777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] PATH_INFO: `/v3/users/830320d05a1b426bac4849fb2f541f8c` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:16.275975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830320d05a1b426bac4849fb2f541f8c)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:16.278615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:16.330370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830320d05a1b426bac4849fb2f541f8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.331096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830320d05a1b426bac4849fb2f541f8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.343674 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830320d05a1b426bac4849fb2f541f8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.344639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830320d05a1b426bac4849fb2f541f8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.351985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830320d05a1b426bac4849fb2f541f8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.364910 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1274/2539] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:38:16 2022] DELETE /identity/v3/users/830320d05a1b426bac4849fb2f541f8c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:38:16.559381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:16.560961 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:16.576563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04dfd5-385b-45f9-a747-55cabb7801bc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:16.630193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:16.638461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:16.639418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:16.639732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:16.640120 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] PATH_INFO: `/v3/projects/52a21e84f1ee4a9c847118c7570d23b4` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:16.641336 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a21e84f1ee4a9c847118c7570d23b4)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:16.643581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:16.696361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a21e84f1ee4a9c847118c7570d23b4', 'request_id': 'req-18a22810-3904-4afb-95ef-9848b6beced1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.706925 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a21e84f1ee4a9c847118c7570d23b4', 'request_id': 'req-18a22810-3904-4afb-95ef-9848b6beced1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.713468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a21e84f1ee4a9c847118c7570d23b4', 'request_id': 'req-18a22810-3904-4afb-95ef-9848b6beced1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.724990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] The token cache is being invalidate because project 52a21e84f1ee4a9c847118c7570d23b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:38:16.724990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.725618 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:16.727193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1266/2540] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:38:16 2022] DELETE /identity/v3/projects/52a21e84f1ee4a9c847118c7570d23b4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:38:26.360812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:26.721651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:37.893099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:37.895277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:37.976129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8532f-0fe2-414d-b120-c7c9d43bc605 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:38.046223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:38.046850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:38.047142 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:38.047413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:38.049429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:38.062385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:38.063995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:38.075104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1275/2541] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:39.638738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:39.640244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:39.703717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a22810-3904-4afb-95ef-9848b6beced1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:39.708456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:39.709148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:39.709430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:39.709708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:39.711700 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:39.791046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:39.793025 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:39.804430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1267/2542] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:43.376856 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:43.378334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:43.393119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0b6e-672e-41ec-b502-efbbd1660336 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:43.399763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:43.400850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:43.401805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:43.402256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:43.404614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:43.420386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:43.422572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:43.437957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1276/2543] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:44.373639 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:44.375202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:44.388684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e4bbd-cb58-46d8-b6eb-245f8784ff5f service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:44.395323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:44.396534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:44.397017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:44.397360 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] PATH_INFO: `/v3/users/26807e5e7aab425eb28e781289794265` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:44.398902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26807e5e7aab425eb28e781289794265)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:44.400921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:44.465463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26807e5e7aab425eb28e781289794265'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.466207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26807e5e7aab425eb28e781289794265'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.480863 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26807e5e7aab425eb28e781289794265'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.481620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26807e5e7aab425eb28e781289794265'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.491809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e246343d-3852-4e35-9a1c-627dc186bd57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26807e5e7aab425eb28e781289794265'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.510507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1268/2544] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:38:44 2022] DELETE /identity/v3/users/26807e5e7aab425eb28e781289794265 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:38:44.700937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:44.716019 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de589ba-dde0-4a34-8e7a-88877c2d283a service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:44.800688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:44.801625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:44.801907 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:44.802183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] PATH_INFO: `/v3/projects/663ec7a59bc147299f904d93d119c5e7` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:44.803417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=663ec7a59bc147299f904d93d119c5e7)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:44.805376 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:44.817049 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:44.855115 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663ec7a59bc147299f904d93d119c5e7', 'request_id': 'req-29822b4c-c877-436b-931c-8afce3533439'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.861273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663ec7a59bc147299f904d93d119c5e7', 'request_id': 'req-29822b4c-c877-436b-931c-8afce3533439'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.867034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663ec7a59bc147299f904d93d119c5e7', 'request_id': 'req-29822b4c-c877-436b-931c-8afce3533439'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.881155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] The token cache is being invalidate because project 663ec7a59bc147299f904d93d119c5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:38:44.881438 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.882066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-29822b4c-c877-436b-931c-8afce3533439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:38:44.883615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1277/2545] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:38:44 2022] DELETE /identity/v3/projects/663ec7a59bc147299f904d93d119c5e7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:38:44.889003 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:44.889287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:44.889560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:44.908075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:44.909697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:44.928370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:44.972777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1269/2546] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:38:44.980802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1739205-6a4d-4752-be89-2caf2d6870a8 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:44.981116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1739205-6a4d-4752-be89-2caf2d6870a8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:44.981405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1739205-6a4d-4752-be89-2caf2d6870a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.001377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e1739205-6a4d-4752-be89-2caf2d6870a8 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:45.002597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e1739205-6a4d-4752-be89-2caf2d6870a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:45.011179 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1278/2547] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:38:45.015492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.025660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecca45-0e8a-44ca-a149-c9ed0b2eb7e1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.030944 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.031658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.032089 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.032485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.033314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:38:45.033541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.035536 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.048069 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.048342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.048612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.054630 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1270/2548] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:38:45.055658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.065489 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712567cf-a249-417c-8693-e0a53785e2e7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.070640 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.071301 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.071596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.071835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.072112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] MFA Rules not processed for user `72419918d6c34080b6a4a4c73b2c06b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:45.072607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.073355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:45.074706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.105527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1271/2549] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:38:45 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:38:45.106591 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.130658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1279/2550] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:38:45.178701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87b39-4314-46b7-83df-487fe747865d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.184219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.184929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.185204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.185448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.186285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.188327 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.211006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1272/2551] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:38:45 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:38:45.215140 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.226181 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a77b3-7c37-4184-a4df-094f49396217 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.230850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.231518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.231798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.232067 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.232824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.234420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.245428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1280/2552] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:38:45.249658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.260216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a547f1-c854-4161-8c15-58267e325b18 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.265088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe604cc-8908-4dd0-8746-c2e6d10185bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.265797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe604cc-8908-4dd0-8746-c2e6d10185bb admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.266082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe604cc-8908-4dd0-8746-c2e6d10185bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.266355 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe604cc-8908-4dd0-8746-c2e6d10185bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.267085 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe604cc-8908-4dd0-8746-c2e6d10185bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.268722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe604cc-8908-4dd0-8746-c2e6d10185bb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.279703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1273/2553] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:38:45.284057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.295784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a27a12-da00-463f-96ae-8fb541a7af60 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.300501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.302057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.302330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.302599 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] PATH_INFO: `/v3/projects/373556d3416e4db68d08897c3d7c0a54/users/7294e5eafdfc42fbac105cf58ee07b70/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.318208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=373556d3416e4db68d08897c3d7c0a54, user_id=7294e5eafdfc42fbac105cf58ee07b70, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.321301 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.337239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1281/2554] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:38:45 2022] PUT /identity/v3/projects/373556d3416e4db68d08897c3d7c0a54/users/7294e5eafdfc42fbac105cf58ee07b70/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:38:45.342460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.342755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.343051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.361310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:45.362533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:45.436480 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1274/2555] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:38:45.447285 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.449126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:45.457402 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:45.459263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:45.470703 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:45.512424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29fff-f362-462a-a971-cb3188f624f7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.516341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.517051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.517335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.517611 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.527975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.529704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.532247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684538b1-a15b-4a86-afe4-bedc444686fc None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:45.537382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:45.538646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:45.539107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:45.539365 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:45.541194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:45.552728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1282/2556] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:45.554516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:45.556034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a862590-82df-474d-a630-e506b1b59101 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:45.571875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1275/2557] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:38:50.340324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:50.351895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ef029-d36e-4ee0-aaef-99c4949bfb7a service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:50.383349 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:50.385010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:50.393378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:50.397220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:50.407184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:50.433273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:38:50.447362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:50.449319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:50.467848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1283/2558] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:50.514728 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db38bc-cc02-4e67-b206-ed7f664a988f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:50.515296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db38bc-cc02-4e67-b206-ed7f664a988f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:50.515527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db38bc-cc02-4e67-b206-ed7f664a988f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:50.546155 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-c6db38bc-cc02-4e67-b206-ed7f664a988f None None] MFA Rules not processed for user `7294e5eafdfc42fbac105cf58ee07b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:50.547435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c6db38bc-cc02-4e67-b206-ed7f664a988f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:50.657509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1276/2559] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:38:50.903763 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:38:50.905521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:51.048161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd7ab0-2a7d-4424-abd0-20d750630e9d service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:38:51.060307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:38:51.061143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:51.061487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:51.061829 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:51.073078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:38:51.074816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:38:51.086193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1284/2560] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:38:52.963981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:38:52.964946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:38:52.965409 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:38:52.996406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] MFA Rules not processed for user `7294e5eafdfc42fbac105cf58ee07b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:38:52.997697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:38:53.010396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1277/2561] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:01.080417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=37 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:02.716832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:02.730113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60861620-9032-4f77-9226-8455ec4c7e04 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:02.736538 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:02.738302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:02.738302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:02.738302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:02.739721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:02.765426 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:02.767053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:02.778261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1285/2562] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:39:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:39:02.988556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=58 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:06.564102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:06.566109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:06.605375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69dae29-c3e7-41f2-b835-34eb0d9a09a4 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:06.610417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:06.611548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:06.611822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:06.612249 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] PATH_INFO: `/v3/users/4bdc0eeb6cd443d78aefefca488d8c5d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:06.615619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdc0eeb6cd443d78aefefca488d8c5d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:06.618040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:06.672540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdc0eeb6cd443d78aefefca488d8c5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:06.673456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdc0eeb6cd443d78aefefca488d8c5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:06.697133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdc0eeb6cd443d78aefefca488d8c5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:06.698583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdc0eeb6cd443d78aefefca488d8c5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:06.714252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fc147f87-dcab-49ee-82a2-6d9915ad9685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdc0eeb6cd443d78aefefca488d8c5d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:06.734178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1278/2563] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:39:06 2022] DELETE /identity/v3/users/4bdc0eeb6cd443d78aefefca488d8c5d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:39:06.943615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:06.956295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c0110-2c45-45f1-8e1f-43d91b670440 service nova] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:07.033127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:07.034608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:07.035171 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:07.035735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] PATH_INFO: `/v3/projects/e3eabfe3692b44d8ae5f7d137ce1d157` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:07.037211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3eabfe3692b44d8ae5f7d137ce1d157)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:07.039353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:07.097269 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3eabfe3692b44d8ae5f7d137ce1d157', 'request_id': 'req-84852e28-5e09-4f40-ba64-aa2a789ecd49'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:07.104568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3eabfe3692b44d8ae5f7d137ce1d157', 'request_id': 'req-84852e28-5e09-4f40-ba64-aa2a789ecd49'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:07.110665 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3eabfe3692b44d8ae5f7d137ce1d157', 'request_id': 'req-84852e28-5e09-4f40-ba64-aa2a789ecd49'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:07.123096 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] The token cache is being invalidate because project e3eabfe3692b44d8ae5f7d137ce1d157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:39:07.123379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:07.123937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-84852e28-5e09-4f40-ba64-aa2a789ecd49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:07.125528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1286/2564] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:39:06 2022] DELETE /identity/v3/projects/e3eabfe3692b44d8ae5f7d137ce1d157 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:39:09.205102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b710-876e-4f18-a2c1-8c0c380e0434 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:09.205684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b710-876e-4f18-a2c1-8c0c380e0434 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:09.205684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b710-876e-4f18-a2c1-8c0c380e0434 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:09.207372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1279/2565] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:39:09.215417 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7ddd5-d5ae-4c8f-b85a-8f7a8e78ddf7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:09.215998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7ddd5-d5ae-4c8f-b85a-8f7a8e78ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:09.215998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7ddd5-d5ae-4c8f-b85a-8f7a8e78ddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:09.235257 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-8ae7ddd5-d5ae-4c8f-b85a-8f7a8e78ddf7 None None] MFA Rules not processed for user `94b91e199622457787fa2a7e78d743c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:09.236672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8ae7ddd5-d5ae-4c8f-b85a-8f7a8e78ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:09.299112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1287/2566] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:39:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:39:09.312667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6b710-876e-4f18-a2c1-8c0c380e0434 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:09.314476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-92f6b710-876e-4f18-a2c1-8c0c380e0434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:09.327732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6b710-876e-4f18-a2c1-8c0c380e0434 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:09.331284 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00747f-6bcb-4008-b442-f97fd63a2380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:09.331937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00747f-6bcb-4008-b442-f97fd63a2380 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:09.332217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00747f-6bcb-4008-b442-f97fd63a2380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:09.332491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00747f-6bcb-4008-b442-f97fd63a2380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:09.340217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00747f-6bcb-4008-b442-f97fd63a2380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:09.341895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00747f-6bcb-4008-b442-f97fd63a2380 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:09.350011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1280/2567] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:39:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:39:09.439283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539e7d0-9f0e-4794-a37e-868fd6daceed None None] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:09.439643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539e7d0-9f0e-4794-a37e-868fd6daceed None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:09.439988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539e7d0-9f0e-4794-a37e-868fd6daceed None None] PATH_INFO: `/` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:09.441460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1288/2568] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:39:09.447589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:09.447980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:09.448348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:09.495775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:09.497046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:09.564971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1281/2569] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:39:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:09.575951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539e7d0-9f0e-4794-a37e-868fd6daceed None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:09.588835 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539e7d0-9f0e-4794-a37e-868fd6daceed None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:09.592526 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:09.593221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:09.593501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:09.593776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:09.605203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:09.606819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:09.618466 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1289/2570] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:39:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:39:19.555348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:19.613057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:39.431882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:39.433491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:39.446253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173600a6-fd52-430e-8fec-f86dec0cff31 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:39.450835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:39.451481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:39.451744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:39.452031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:39.454442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:39.521060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:39.522569 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:39.533544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1282/2571] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:39:42.744232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:42.746099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:42.764677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043538-149b-433c-9c32-9a4393fb2f51 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:42.770515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:42.771854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:42.772139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:42.772418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] PATH_INFO: `/v3/users/6ddb38b4d6864a6c96b84c9a4092eea8` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:42.773750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddb38b4d6864a6c96b84c9a4092eea8)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:42.775697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:42.832930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddb38b4d6864a6c96b84c9a4092eea8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:42.833778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddb38b4d6864a6c96b84c9a4092eea8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:42.846023 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddb38b4d6864a6c96b84c9a4092eea8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:42.846719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddb38b4d6864a6c96b84c9a4092eea8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:42.857343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-c0db0797-60ba-4e53-9ba5-6f3f4389ce08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddb38b4d6864a6c96b84c9a4092eea8'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:42.870475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1290/2572] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:39:42 2022] DELETE /identity/v3/users/6ddb38b4d6864a6c96b84c9a4092eea8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:39:43.058464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.072825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd498-9ffc-4ee3-8225-1066e9327da2 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.138889 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.139936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.140043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.140304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] PATH_INFO: `/v3/projects/b60d10dba1d947cc97149a063a810129` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.141534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60d10dba1d947cc97149a063a810129)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.144923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.202011 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60d10dba1d947cc97149a063a810129', 'request_id': 'req-b569c2bd-09a0-4e9e-95de-f28aabd53d15'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:43.222169 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60d10dba1d947cc97149a063a810129', 'request_id': 'req-b569c2bd-09a0-4e9e-95de-f28aabd53d15'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:43.230012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60d10dba1d947cc97149a063a810129', 'request_id': 'req-b569c2bd-09a0-4e9e-95de-f28aabd53d15'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:43.248042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] The token cache is being invalidate because project b60d10dba1d947cc97149a063a810129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:39:43.248430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:43.249433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b569c2bd-09a0-4e9e-95de-f28aabd53d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:39:43.251502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1283/2573] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:39:43 2022] DELETE /identity/v3/projects/b60d10dba1d947cc97149a063a810129 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:39:43.258051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.258453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.258604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.277574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:43.279113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:43.340771 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1291/2574] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:43.350320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.350607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.350880 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.369617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:43.370822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:43.379374 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1284/2575] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:43.385597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.400346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a986705-1fdb-4eab-bad3-0a29a1d76b21 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.408566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.409496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.409878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.410253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.411383 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:39:43.411810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.413593 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.425498 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1292/2576] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:39:43.429843 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.439456 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d0d37-c6ba-43b5-8d55-afd35e4f8dae None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.445147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.445795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.446074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.446346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.447032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.448982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.464483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:43.489810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1285/2577] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:39:43 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:39:43.494229 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.504528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba8458-e422-4b10-a7a9-4ff4566e4b7d admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.570809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.571531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.571812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.572084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.572831 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.574802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.581649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:43.599937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1293/2578] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:39:43 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:39:43.603929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.614200 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448c918-71f4-4793-b4a9-9d43ab9ff0e4 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.619257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.619916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.620193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.620469 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.621224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.622831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.633413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1286/2579] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:39:43.637457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.649071 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebfd52-a79f-4582-b522-34f37a343442 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.653944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd432961-8339-4cc0-b50b-b31465c072b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.654713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd432961-8339-4cc0-b50b-b31465c072b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.655070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd432961-8339-4cc0-b50b-b31465c072b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.655379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd432961-8339-4cc0-b50b-b31465c072b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.656104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd432961-8339-4cc0-b50b-b31465c072b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.658110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd432961-8339-4cc0-b50b-b31465c072b7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.670481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1294/2580] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:39:43.675117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.686058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcffa5-ab4e-4703-a374-e9d4adf87f39 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.691581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.692737 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.693055 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.693337 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] PATH_INFO: `/v3/projects/359201313a944a8fa9881ee6fdea67e2/users/e935f2637d124453947b3c78360567a5/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.710991 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=359201313a944a8fa9881ee6fdea67e2, user_id=e935f2637d124453947b3c78360567a5, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.713711 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.727176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1287/2581] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:39:43 2022] PUT /identity/v3/projects/359201313a944a8fa9881ee6fdea67e2/users/e935f2637d124453947b3c78360567a5/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:39:43.732293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21a5c9-e447-418d-9aff-18089851730c None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.732584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21a5c9-e447-418d-9aff-18089851730c None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.732895 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21a5c9-e447-418d-9aff-18089851730c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.751727 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9f21a5c9-e447-418d-9aff-18089851730c None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:43.752939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9f21a5c9-e447-418d-9aff-18089851730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:43.813774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1295/2582] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:43.824228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:43.826245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:43.887267 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b7c84-1821-4fe2-b8e0-29b853afcfeb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:43.891045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:43.891707 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:43.891987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:43.892261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:43.902324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:43.903755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:43.914113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1288/2583] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:39:47.982196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69017631-244b-40ea-8cfe-4558e5f5544b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:47.982692 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69017631-244b-40ea-8cfe-4558e5f5544b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:47.982797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69017631-244b-40ea-8cfe-4558e5f5544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:48.016973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-69017631-244b-40ea-8cfe-4558e5f5544b None None] MFA Rules not processed for user `e935f2637d124453947b3c78360567a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:48.018202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-69017631-244b-40ea-8cfe-4558e5f5544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:48.092305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1296/2584] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:48.106693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:39:48.108649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:48.210502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecbd76-110e-4364-939b-505d560e4590 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:39:48.216756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ddf8d-934b-4e31-bb71-fbf3afa137ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:39:48.217679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ddf8d-934b-4e31-bb71-fbf3afa137ed service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:48.218118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ddf8d-934b-4e31-bb71-fbf3afa137ed service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:48.218553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ddf8d-934b-4e31-bb71-fbf3afa137ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:48.228002 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ddf8d-934b-4e31-bb71-fbf3afa137ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:39:48.229592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ddf8d-934b-4e31-bb71-fbf3afa137ed service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:39:48.239596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1289/2585] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:39:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:39:49.541722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:39:49.542252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:39:49.542252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:39:49.560858 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] MFA Rules not processed for user `e935f2637d124453947b3c78360567a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:39:49.561876 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:39:49.569459 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1297/2586] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:39:58.235121 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:39:59.555084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:00.463172 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c86f-8d4d-4723-ab5e-a9116e76c910 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:00.463739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c86f-8d4d-4723-ab5e-a9116e76c910 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:00.463739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c86f-8d4d-4723-ab5e-a9116e76c910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:00.481795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-b530c86f-8d4d-4723-ab5e-a9116e76c910 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:40:00.483042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-b530c86f-8d4d-4723-ab5e-a9116e76c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:00.544557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1290/2587] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:40:00.597373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:00.599591 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:00.677805 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea4e0-0259-4e07-a766-b3720ccbe099 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:00.683412 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906773f5-3db1-4e1d-b4fc-9ae5da1a943a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:00.684100 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906773f5-3db1-4e1d-b4fc-9ae5da1a943a service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:00.684475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906773f5-3db1-4e1d-b4fc-9ae5da1a943a service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:00.684650 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906773f5-3db1-4e1d-b4fc-9ae5da1a943a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:00.693673 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906773f5-3db1-4e1d-b4fc-9ae5da1a943a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:00.695281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906773f5-3db1-4e1d-b4fc-9ae5da1a943a service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:00.704089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1298/2588] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:40:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:40:10.536713 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=25 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:10.700890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=32 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:13.149145 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e909-27ee-4322-943e-cf2be2b96bd7 None None] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:13.149729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e909-27ee-4322-943e-cf2be2b96bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:13.149729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688e909-27ee-4322-943e-cf2be2b96bd7 None None] PATH_INFO: `/` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:13.151152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1291/2589] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jun 30 12:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:40:13.160377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88f7f-d7ca-4d64-90a9-60b3d3aaa0f1 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:13.160377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88f7f-d7ca-4d64-90a9-60b3d3aaa0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:13.160377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88f7f-d7ca-4d64-90a9-60b3d3aaa0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:13.178773 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-a1b88f7f-d7ca-4d64-90a9-60b3d3aaa0f1 None None] MFA Rules not processed for user `0ca71f32f61e41288cac452e58b4a1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:40:13.180745 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-a1b88f7f-d7ca-4d64-90a9-60b3d3aaa0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:13.193905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1299/2590] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jun 30 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:40:13.207174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e909-27ee-4322-943e-cf2be2b96bd7 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:13.209150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7688e909-27ee-4322-943e-cf2be2b96bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:13.272815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688e909-27ee-4322-943e-cf2be2b96bd7 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:13.276358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e81a1c18d34adf8d8a45fa09f26099', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:13.277052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:13.277333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:13.277608 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:13.288963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:13.300619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:13.312111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1292/2591] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:40:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:40:16.014703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869cdef-7645-4daf-95aa-86e43c00c079 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:16.015227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869cdef-7645-4daf-95aa-86e43c00c079 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:16.015227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869cdef-7645-4daf-95aa-86e43c00c079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:16.035272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-b869cdef-7645-4daf-95aa-86e43c00c079 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:40:16.036513 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b869cdef-7645-4daf-95aa-86e43c00c079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:16.044823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1300/2592] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:40:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:40:16.058054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:16.070490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5f1ce-1d9f-4186-9339-6c0af2060180 service glance] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:16.076517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:16.077306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:16.077494 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:16.077755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:16.086335 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:16.087939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:16.096220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1293/2593] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:40:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 12:40:26.029889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:26.092866 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=32 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:34.846283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:34.846815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:34.846815 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:34.865921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:40:34.867155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:34.874564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1301/2594] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jun 30 12:40:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:40:34.886816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:34.897717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e3eb-f334-43bb-988a-1bfa17451679 service swift] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:34.902948 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d39b6-d44f-4b97-a3f1-53e33834ea5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:34.903605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d39b6-d44f-4b97-a3f1-53e33834ea5d service swift] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:34.903888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d39b6-d44f-4b97-a3f1-53e33834ea5d service swift] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:34.904533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d39b6-d44f-4b97-a3f1-53e33834ea5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:34.913038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d39b6-d44f-4b97-a3f1-53e33834ea5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:34.914628 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d39b6-d44f-4b97-a3f1-53e33834ea5d service swift] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:34.922917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1294/2595] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:40:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:40:44.860103 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:44.919508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=6 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:40:47.836341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:47.838598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:47.897709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43661d-60a1-44ee-8f88-1c6144e7c620 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:47.902298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:47.902947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:47.903223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:47.903872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:47.905834 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:47.966053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:47.967506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:47.977602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1302/2596] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:40:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:40:49.787568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c793-5711-455a-9e0b-bad50826325d None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:49.788052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c793-5711-455a-9e0b-bad50826325d None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:49.788181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c793-5711-455a-9e0b-bad50826325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:49.806541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-48c1c793-5711-455a-9e0b-bad50826325d None None] MFA Rules not processed for user `e935f2637d124453947b3c78360567a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:40:49.807750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-48c1c793-5711-455a-9e0b-bad50826325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:49.818086 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1295/2597] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:40:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:40:50.595887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:50.597344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:50.609012 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326ff5e-18fc-4a50-95a4-fe9600f20f4b service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:50.613708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:50.614344 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:50.614639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:50.614911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:50.616799 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:50.680780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:50.682290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:50.691676 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1303/2598] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:40:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:40:54.980839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:54.981340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:54.981340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:55.001315 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] MFA Rules not processed for user `72419918d6c34080b6a4a4c73b2c06b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:40:55.002529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:55.010095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1296/2599] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:40:58.058705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:40:58.070113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be023a07-7d18-45d7-b513-e1a4cf7decf3 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:40:58.074359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:40:58.075224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:40:58.075512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:40:58.075788 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:40:58.077634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:40:58.090759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:40:58.092231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:40:58.104134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1304/2600] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:41:02.214769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:41:02.216298 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:02.293524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f867a4-64d8-4148-b716-c6118cdd2320 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:41:02.299228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:41:02.299890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:02.300174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:02.300448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:02.302475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:02.314623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:41:02.316224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:41:02.327081 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1297/2601] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:41:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:41:06.549822 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:41:06.551927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:06.565721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15fd7d-377d-479e-ac64-8197bb68bf6a service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:41:06.570503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:41:06.571388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:06.571671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:06.572038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] PATH_INFO: `/v3/users/e81aa9d47cd1470abef31a27839eaf86` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:06.573419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81aa9d47cd1470abef31a27839eaf86)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:41:06.575410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:41:06.624219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:41:06.639034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81aa9d47cd1470abef31a27839eaf86'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:06.640495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81aa9d47cd1470abef31a27839eaf86'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:06.654491 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81aa9d47cd1470abef31a27839eaf86'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:06.655196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81aa9d47cd1470abef31a27839eaf86'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:06.663076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81aa9d47cd1470abef31a27839eaf86'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:06.675248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1305/2602] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:41:06 2022] DELETE /identity/v3/users/e81aa9d47cd1470abef31a27839eaf86 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:41:06.894253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:41:06.913268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3266d-c56f-4161-8a1e-e8b3ebe62a73 service placement] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:41:06.985585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:41:06.987516 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:06.987903 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:06.988244 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] PATH_INFO: `/v3/projects/e8abe616c14a4920b1cbc45733fa5771` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:06.990179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8abe616c14a4920b1cbc45733fa5771)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:41:06.992857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:41:07.061678 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8abe616c14a4920b1cbc45733fa5771', 'request_id': 'req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.079432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8abe616c14a4920b1cbc45733fa5771', 'request_id': 'req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.092328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8abe616c14a4920b1cbc45733fa5771', 'request_id': 'req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.112320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] The token cache is being invalidate because project e8abe616c14a4920b1cbc45733fa5771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:41:07.112617 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.113320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.115278 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1298/2603] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:41:06 2022] DELETE /identity/v3/projects/e8abe616c14a4920b1cbc45733fa5771 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:41:07.119163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:41:07.121449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:07.135874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8452b3-c5ff-4f85-8475-28eac8155681 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:41:07.229119 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:41:07.230076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:07.230483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:07.230854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] PATH_INFO: `/v3/users/22c36e462dda4424b81718b0146021a6` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:07.232265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c36e462dda4424b81718b0146021a6)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:41:07.234453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:41:07.292353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c36e462dda4424b81718b0146021a6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.293033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c36e462dda4424b81718b0146021a6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.305961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c36e462dda4424b81718b0146021a6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.306985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c36e462dda4424b81718b0146021a6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.317743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c36e462dda4424b81718b0146021a6'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.339082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1306/2604] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:41:07 2022] DELETE /identity/v3/users/22c36e462dda4424b81718b0146021a6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:41:07.513590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:41:07.515090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:07.531111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fd299-def8-4b0c-a8bd-310d06ca3a4d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:41:07.536377 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:41:07.593727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:41:07.594644 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:07.594940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:07.595211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] PATH_INFO: `/v3/projects/401bb3c5eb1b49eb8f2aed9166673be4` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:07.596441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401bb3c5eb1b49eb8f2aed9166673be4)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:41:07.598413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:41:07.647939 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401bb3c5eb1b49eb8f2aed9166673be4', 'request_id': 'req-9d5be89f-47c0-4ff4-a70f-9d663879e722'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.657053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401bb3c5eb1b49eb8f2aed9166673be4', 'request_id': 'req-9d5be89f-47c0-4ff4-a70f-9d663879e722'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.662906 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401bb3c5eb1b49eb8f2aed9166673be4', 'request_id': 'req-9d5be89f-47c0-4ff4-a70f-9d663879e722'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.674048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] The token cache is being invalidate because project 401bb3c5eb1b49eb8f2aed9166673be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:41:07.674328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.674968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-9d5be89f-47c0-4ff4-a70f-9d663879e722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:41:07.676648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1299/2605] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:41:07 2022] DELETE /identity/v3/projects/401bb3c5eb1b49eb8f2aed9166673be4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:41:17.335081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=69,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:41:17.671279 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:41:19.095929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:41:19.097520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:19.169154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a3d09-89e7-4007-82ef-ebff15f8e20f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:41:19.230052 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:41:19.230912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:19.231193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:19.231479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:19.233364 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:19.245700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:41:19.247357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8a300-1d58-47ee-ac3d-cad408b8ed1a service placement] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:41:19.257906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1307/2606] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:41:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:41:22.589825 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:22.590406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:22.590406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:22.614232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] MFA Rules not processed for user `7294e5eafdfc42fbac105cf58ee07b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:41:22.615460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:22.680849 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1300/2607] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:41:29.253475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=58 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:41:32.673107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=25 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:41:54.364963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:41:54.365464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:41:54.365564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:41:54.385277 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] MFA Rules not processed for user `e935f2637d124453947b3c78360567a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:41:54.386507 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:41:54.443248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1308/2608] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:41:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:42:04.436370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=25 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:42:54.005581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:42:54.007251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:42:54.076781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb140b-30f3-48e9-85ce-4b9955465d45 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:42:54.080685 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:42:54.081389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:42:54.081673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:42:54.081946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:42:54.083859 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:42:54.167502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:42:54.169004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42490626-4fa4-4ae7-9851-875c9f419d7c service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:42:54.180966 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1301/2609] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:42:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:42:56.730894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:42:56.741703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d150c5e-f284-4ccf-a934-8809135cacac None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:42:56.745420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:42:56.746102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:42:56.746379 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:42:56.746648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:42:56.748715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:42:56.768009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:42:56.770279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:42:56.800113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1309/2610] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:42:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:43:01.361461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:01.362004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:01.362004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:01.379946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] MFA Rules not processed for user `e935f2637d124453947b3c78360567a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:43:01.381222 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:01.388928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1302/2611] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:43:06.780174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:08.281015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:08.290618 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cde4dd-dc29-40b2-9632-2b8362456acd service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:08.294812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:08.295495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:08.295778 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:08.296054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:08.298390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:08.364193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:08.365728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:08.375622 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1310/2612] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:43:11.374414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=57 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:16.191550 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:16.194649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:16.218641 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f7f1-f9c9-4d1e-98ae-b7c19ee72ee5 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:16.226549 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:16.227467 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:16.227754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:16.228035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] PATH_INFO: `/v3/users/72419918d6c34080b6a4a4c73b2c06b2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:16.229296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72419918d6c34080b6a4a4c73b2c06b2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:16.231318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:16.288656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72419918d6c34080b6a4a4c73b2c06b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.289356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72419918d6c34080b6a4a4c73b2c06b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.315904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72419918d6c34080b6a4a4c73b2c06b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.319585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72419918d6c34080b6a4a4c73b2c06b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.329290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72419918d6c34080b6a4a4c73b2c06b2'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.344324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1303/2613] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:43:16 2022] DELETE /identity/v3/users/72419918d6c34080b6a4a4c73b2c06b2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:43:16.534203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:16.550704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59814ef6-6df3-4088-aba2-368148d1fd0a service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:16.617339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:16.618327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:16.618686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:16.619165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] PATH_INFO: `/v3/projects/c7b272349089480984fd1c5eabe6dce9` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:16.620462 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b272349089480984fd1c5eabe6dce9)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:16.622485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:16.675557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b272349089480984fd1c5eabe6dce9', 'request_id': 'req-2416fe3f-e403-4967-bdc9-e89f71a6bbda'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.682209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b272349089480984fd1c5eabe6dce9', 'request_id': 'req-2416fe3f-e403-4967-bdc9-e89f71a6bbda'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.687848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b272349089480984fd1c5eabe6dce9', 'request_id': 'req-2416fe3f-e403-4967-bdc9-e89f71a6bbda'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.704605 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] The token cache is being invalidate because project c7b272349089480984fd1c5eabe6dce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:43:16.704951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.705784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.707377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1311/2614] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:43:16 2022] DELETE /identity/v3/projects/c7b272349089480984fd1c5eabe6dce9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:43:16.711399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:16.712943 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:16.726725 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734507e-d981-41c3-8cb5-328feee04f41 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:16.784270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:16.785210 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:16.785496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:16.785757 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] PATH_INFO: `/v3/users/ca9f038b657a4328869a9c3499d7ea1d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:16.786965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9f038b657a4328869a9c3499d7ea1d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:16.788940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:16.842349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9f038b657a4328869a9c3499d7ea1d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.843148 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9f038b657a4328869a9c3499d7ea1d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.861302 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9f038b657a4328869a9c3499d7ea1d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.862106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9f038b657a4328869a9c3499d7ea1d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.871243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f56aa3fa-90b5-4e01-93b8-30b077cb886a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9f038b657a4328869a9c3499d7ea1d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:16.873784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:16.888387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1304/2615] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:43:16 2022] DELETE /identity/v3/users/ca9f038b657a4328869a9c3499d7ea1d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:43:17.073182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:17.074709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:17.087701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416fe3f-e403-4967-bdc9-e89f71a6bbda admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:17.115106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:17.153790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:17.154647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:17.154927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:17.155200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] PATH_INFO: `/v3/projects/15962efa920b497b88991a1d1f1b26c6` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:17.156418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15962efa920b497b88991a1d1f1b26c6)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:17.158572 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:17.213545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15962efa920b497b88991a1d1f1b26c6', 'request_id': 'req-2102ece4-93bc-4c78-beb2-06d79371df69'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:17.219982 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15962efa920b497b88991a1d1f1b26c6', 'request_id': 'req-2102ece4-93bc-4c78-beb2-06d79371df69'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:17.225917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15962efa920b497b88991a1d1f1b26c6', 'request_id': 'req-2102ece4-93bc-4c78-beb2-06d79371df69'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:17.239912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] The token cache is being invalidate because project 15962efa920b497b88991a1d1f1b26c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:43:17.240196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:17.240892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:43:17.242519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1312/2616] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:43:17 2022] DELETE /identity/v3/projects/15962efa920b497b88991a1d1f1b26c6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:43:18.366690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:18.367264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:18.367264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:18.388369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:43:18.390129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:18.454026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1305/2617] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:43:18.503551 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:18.505199 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:18.577708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ece4-93bc-4c78-beb2-06d79371df69 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:18.583453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:18.584104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:18.584385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:18.584708 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:18.593042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:18.594638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:18.602779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1313/2618] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:43:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:43:28.447104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=29 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:28.599756 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:32.193014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:32.194601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:32.262873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea646fb-0ebe-4a39-9c8b-84e6e5f55f70 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:32.267100 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:32.267755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:32.268038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:32.268311 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:32.270533 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:32.281646 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:32.283083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1872f67-54c7-4639-8c9d-25ccfc4749dd service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:32.291883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1306/2619] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:43:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:43:32.586926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:32.588500 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:32.663497 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b39e4-c816-4280-bf71-425069c476ab service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:32.669505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:32.670159 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:32.670440 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:32.670711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:32.672824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:32.684713 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:32.686345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:32.696600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1314/2620] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:43:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:43:34.591596 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070b14d-7bf6-4fe2-b339-bebcc7aa3e70 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:34.592105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070b14d-7bf6-4fe2-b339-bebcc7aa3e70 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:34.592212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070b14d-7bf6-4fe2-b339-bebcc7aa3e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:34.617774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8070b14d-7bf6-4fe2-b339-bebcc7aa3e70 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:43:34.618973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8070b14d-7bf6-4fe2-b339-bebcc7aa3e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:34.626548 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1307/2621] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:43:34.639607 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:34.652779 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09515c-d13d-4f67-85c6-7dfa8838e787 service placement] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:34.661079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:34.662182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:34.662433 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:34.662707 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:34.671639 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:34.673323 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:34.681880 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1315/2622] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:43:44.611706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=29 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:44.678546 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=38 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:43:50.018782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:50.019321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:50.019321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:50.039890 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:43:50.041165 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:50.048721 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1308/2623] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jun 30 12:43:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:43:50.062046 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:50.073407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5519f0-17dd-442f-a382-066314c65e0e service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:50.078843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:50.079481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:50.079758 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:50.080028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:50.088048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:50.089669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:50.097816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1316/2624] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:43:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:43:51.454288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:43:51.456319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:51.538340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa7701-38e5-436b-88e9-bae9783186ec None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:43:51.544560 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:43:51.545246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:43:51.545528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:43:51.545798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:43:51.547741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:43:51.608501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:43:51.610111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:43:51.619626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1309/2625] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:43:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:00.094335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:01.614923 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=59 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:01.952037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:01.954287 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:01.968236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44f82-f947-4712-9a9e-821826599d07 service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:01.973318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:01.974008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:01.974289 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:01.974560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:01.977116 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:01.989184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:01.990619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:01.998862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1317/2626] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:05.401243 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:05.413950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b480b98-a7df-4b00-bec3-55c5e3498674 service nova] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:05.424598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:05.425307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:05.425622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:05.426532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:05.429281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:05.443072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:05.444734 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:05.459212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1310/2627] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:11.995087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:13.806740 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:13.809085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:13.876777 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc31bb-388d-444e-82df-63e3d56a1e66 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:13.885687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:13.887025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:13.887374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:13.887644 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] PATH_INFO: `/v3/users/e935f2637d124453947b3c78360567a5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:13.889769 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e935f2637d124453947b3c78360567a5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:13.893073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:13.947542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e935f2637d124453947b3c78360567a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:13.948157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e935f2637d124453947b3c78360567a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:13.960352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e935f2637d124453947b3c78360567a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:13.961070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e935f2637d124453947b3c78360567a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:13.970121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3298298c-411e-40da-ab18-061ceab5eab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e935f2637d124453947b3c78360567a5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:13.983535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1318/2628] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:13 2022] DELETE /identity/v3/users/e935f2637d124453947b3c78360567a5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:14.174476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.185926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfff08-af8e-4d7a-9d32-63c54b32285b service nova] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.245259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.246131 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.246415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.246712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] PATH_INFO: `/v3/projects/359201313a944a8fa9881ee6fdea67e2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.247909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=359201313a944a8fa9881ee6fdea67e2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.249883 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.299040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359201313a944a8fa9881ee6fdea67e2', 'request_id': 'req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:14.310741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359201313a944a8fa9881ee6fdea67e2', 'request_id': 'req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:14.319807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359201313a944a8fa9881ee6fdea67e2', 'request_id': 'req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:14.332732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] The token cache is being invalidate because project 359201313a944a8fa9881ee6fdea67e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:14.333037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:14.333802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-fe62571f-9b00-4f80-871b-66f3c2d2a2ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:14.335650 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1311/2629] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:14 2022] DELETE /identity/v3/projects/359201313a944a8fa9881ee6fdea67e2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:14.340823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.341113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.341393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.360357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:14.361629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:14.436386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1319/2630] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:14.447969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.448460 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.448689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.472211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:14.473381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:14.483074 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1312/2631] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:14.487590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.498774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38527b69-4645-4950-a5e8-66b3d9dbc890 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.505331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.505997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.506274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.506541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.507265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:14.507566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.509222 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.522938 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1320/2632] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:14.529712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.544947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db19e1-be04-4c52-b1e7-33ff7bfb5202 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.552755 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.553910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.554232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.554507 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.555202 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.557224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.586934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1313/2633] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:14 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:14.591373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.602057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898c6c-0e2d-4eda-ba3a-2a618306dadf admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.606083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:14.663704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.664389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.664670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.664978 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.665709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.667638 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.689585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1321/2634] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:14 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:14.694448 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.708770 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5b35b-0dd3-4f14-8fab-aa2cd32be5bb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.713926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.714592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.714871 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.715146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.715876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.717520 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.728778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1314/2635] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:14.732560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.742944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1a23-9a6f-4be2-8617-5eedb994b4b8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.748051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a301a3e-b9fc-4184-81cc-f60001dabd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.748746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a301a3e-b9fc-4184-81cc-f60001dabd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.749027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a301a3e-b9fc-4184-81cc-f60001dabd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.749300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a301a3e-b9fc-4184-81cc-f60001dabd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.750031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a301a3e-b9fc-4184-81cc-f60001dabd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.751626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a301a3e-b9fc-4184-81cc-f60001dabd6c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.761843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1322/2636] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:14.765952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.777073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c947da-624a-4a66-b7ef-6bc3c3ec20a2 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:14.782772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:14.783668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.784070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.784352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] PATH_INFO: `/v3/projects/b19a7b71fed24f17b11e35b7de8246b0/users/72db77e36d4d45d3aadf06ef76b143f5/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.799411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19a7b71fed24f17b11e35b7de8246b0, user_id=72db77e36d4d45d3aadf06ef76b143f5, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:14.802093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:14.817008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1315/2637] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:14 2022] PUT /identity/v3/projects/b19a7b71fed24f17b11e35b7de8246b0/users/72db77e36d4d45d3aadf06ef76b143f5/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:14.828719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:14.829235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:14.829359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:14.863294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] MFA Rules not processed for user `72db77e36d4d45d3aadf06ef76b143f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:14.864584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:14.932446 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1323/2638] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:14.937091 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:14.950999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:15.000692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a642c-2b67-4196-aacc-c3db1624d5ee admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.005491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.006152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.006432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.006796 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.007462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.009403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.038343 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1316/2639] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:14 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:15.042889 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.054365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7916ba7-ac43-42d7-8121-b6fadf046702 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.112403 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.113122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.113399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.113675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.114367 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.116401 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.145743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1324/2640] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:15 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:15.150860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.161369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42895fe0-5b33-4b31-afcd-b596872205a7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.167013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.167692 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.167968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.168240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.169029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.170668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.185275 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1317/2641] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:15.189625 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.196130 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:15.200234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc0a59-bd70-47fc-a53f-946ab4a8e163 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.206149 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.207040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.207313 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.207590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] PATH_INFO: `/v3/projects/87b7ed3763c446f684f198ecc096c4eb/users/d9b2e464713f44de9e5d4aac84691a9b/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.222844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b7ed3763c446f684f198ecc096c4eb, user_id=d9b2e464713f44de9e5d4aac84691a9b, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.225597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.241691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1325/2642] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:15 2022] PUT /identity/v3/projects/87b7ed3763c446f684f198ecc096c4eb/users/d9b2e464713f44de9e5d4aac84691a9b/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:15.245744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.255934 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb70de-518b-47ca-8a37-020e52821fb1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.314162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfa025-5daa-414a-9a7f-fdffacf5bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.314829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfa025-5daa-414a-9a7f-fdffacf5bbce admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.315109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfa025-5daa-414a-9a7f-fdffacf5bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.315380 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfa025-5daa-414a-9a7f-fdffacf5bbce admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.316102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfa025-5daa-414a-9a7f-fdffacf5bbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.317769 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfa025-5daa-414a-9a7f-fdffacf5bbce admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.328080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1318/2643] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:15.332517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.344331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772a230-debb-4ec5-a078-2ea6d654803d admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.350251 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.351331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.351609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.351886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] PATH_INFO: `/v3/projects/87b7ed3763c446f684f198ecc096c4eb/users/d9b2e464713f44de9e5d4aac84691a9b/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.354166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b7ed3763c446f684f198ecc096c4eb, user_id=d9b2e464713f44de9e5d4aac84691a9b, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.356863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.368743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1326/2644] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:15 2022] PUT /identity/v3/projects/87b7ed3763c446f684f198ecc096c4eb/users/d9b2e464713f44de9e5d4aac84691a9b/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:15.379019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778763ef-9667-467b-bac3-8753a61de19b None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.379306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778763ef-9667-467b-bac3-8753a61de19b None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.379582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778763ef-9667-467b-bac3-8753a61de19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.409340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-778763ef-9667-467b-bac3-8753a61de19b None None] MFA Rules not processed for user `d9b2e464713f44de9e5d4aac84691a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:15.410576 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-778763ef-9667-467b-bac3-8753a61de19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:15.481786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1319/2645] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:15.494827 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.497851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:15.565292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7b94d-8a9b-49d9-ad83-7815643725a3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.570606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.571262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.571536 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.571806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.582425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.583848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.595099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1327/2646] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:15.930875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778763ef-9667-467b-bac3-8753a61de19b None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.940962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778763ef-9667-467b-bac3-8753a61de19b None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.946653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b2e464713f44de9e5d4aac84691a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b7ed3763c446f684f198ecc096c4eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.947346 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.947633 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.947910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.948748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:15.949070 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.950691 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:15.953296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:15.964762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1320/2647] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:15.969113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:15.978765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f7aad-3d7d-4100-9b0f-4d4273bf76ee service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:15.984657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b2e464713f44de9e5d4aac84691a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b7ed3763c446f684f198ecc096c4eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:15.985324 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:15.985612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:15.985892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:15.986577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:15.988542 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:16.018485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1328/2648] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:15 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:16.022908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:16.033695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c3d4f-0c53-4d91-b2e0-ac8bb16bf0da tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:16.098105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b2e464713f44de9e5d4aac84691a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b7ed3763c446f684f198ecc096c4eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:16.098739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:16.099027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:16.099310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:16.100048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:16.100363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:16.102118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:16.116781 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1321/2649] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:44:16.154612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:16.165232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39621c5-f252-4569-b577-f12eb59a0c7a tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:16.170238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9b2e464713f44de9e5d4aac84691a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b7ed3763c446f684f198ecc096c4eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:16.171125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:16.171399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:16.171677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] PATH_INFO: `/v3/projects/9d616fcbc116408295de8d14a246be78` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:16.172983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9d616fcbc116408295de8d14a246be78)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:16.174894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:16.226026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d616fcbc116408295de8d14a246be78', 'request_id': 'req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.232227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d616fcbc116408295de8d14a246be78', 'request_id': 'req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.237905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d616fcbc116408295de8d14a246be78', 'request_id': 'req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.251694 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] The token cache is being invalidate because project 9d616fcbc116408295de8d14a246be78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:16.251980 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.252620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-3087be4c-6f22-4542-9ea3-eb862a4a6a9f tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.254220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1329/2650] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:16 2022] DELETE /identity/v3/projects/9d616fcbc116408295de8d14a246be78 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:16.453704 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:16.455565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:16.521414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f432397-b997-411f-91ad-e29adf1ecb1b tempest-VolumeQuotaClassesTest-757036052 tempest-VolumeQuotaClassesTest-757036052-project-admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:16.578853 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:16.579708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:16.579985 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:16.580255 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] PATH_INFO: `/v3/users/72db77e36d4d45d3aadf06ef76b143f5` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:16.581632 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72db77e36d4d45d3aadf06ef76b143f5)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:16.583554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:16.626378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:16.633381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72db77e36d4d45d3aadf06ef76b143f5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.634111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72db77e36d4d45d3aadf06ef76b143f5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.646509 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72db77e36d4d45d3aadf06ef76b143f5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.647143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72db77e36d4d45d3aadf06ef76b143f5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.656013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72db77e36d4d45d3aadf06ef76b143f5'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:16.669029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1322/2651] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:16 2022] DELETE /identity/v3/users/72db77e36d4d45d3aadf06ef76b143f5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:16.673830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:16.674122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:16.674400 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:16.694151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:16.695700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:16.721340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:16.758123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1330/2652] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:16.768050 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:16.769664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:16.829428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab7c8f-a62a-41a4-8ff3-2b68a836192a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:16.833087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:16.833736 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:16.834013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:16.834283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:16.843862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:16.845329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:16.855238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1323/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:17.139155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:17.140653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:17.153994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5c840-d4aa-4aa0-9b3c-bc7abd5cacba None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:17.159123 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:17.159997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:17.160316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:17.160568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] PATH_INFO: `/v3/projects/b19a7b71fed24f17b11e35b7de8246b0` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:17.161830 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19a7b71fed24f17b11e35b7de8246b0)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:17.163813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:17.220792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19a7b71fed24f17b11e35b7de8246b0', 'request_id': 'req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.227033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19a7b71fed24f17b11e35b7de8246b0', 'request_id': 'req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.233172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19a7b71fed24f17b11e35b7de8246b0', 'request_id': 'req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.247503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] The token cache is being invalidate because project b19a7b71fed24f17b11e35b7de8246b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:17.247781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.248431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.250050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1331/2654] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:17 2022] DELETE /identity/v3/projects/b19a7b71fed24f17b11e35b7de8246b0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:17.254158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:17.264375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba3687-41cd-4c3d-b303-d2b218fbcde0 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:17.325852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:17.326729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:17.327006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:17.327274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] PATH_INFO: `/v3/users/d9b2e464713f44de9e5d4aac84691a9b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:17.328592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b2e464713f44de9e5d4aac84691a9b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:17.330584 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:17.380352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b2e464713f44de9e5d4aac84691a9b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.381112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b2e464713f44de9e5d4aac84691a9b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.391874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:17.394157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b2e464713f44de9e5d4aac84691a9b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.395060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b2e464713f44de9e5d4aac84691a9b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.405399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-655b065e-0870-4ad7-98be-bcafcb7e2104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b2e464713f44de9e5d4aac84691a9b'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.418956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1324/2655] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:17 2022] DELETE /identity/v3/users/d9b2e464713f44de9e5d4aac84691a9b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:17.701627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:17.703206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:17.719352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b62b5-f8f1-42e1-a9b4-9bd8d86070c1 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:17.779192 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:17.780054 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:17.780332 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:17.780598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] PATH_INFO: `/v3/projects/87b7ed3763c446f684f198ecc096c4eb` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:17.781832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b7ed3763c446f684f198ecc096c4eb)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:17.783739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:17.835253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b7ed3763c446f684f198ecc096c4eb', 'request_id': 'req-6ddc7efb-5861-40db-a794-ed33c350dc43'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.841735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b7ed3763c446f684f198ecc096c4eb', 'request_id': 'req-6ddc7efb-5861-40db-a794-ed33c350dc43'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.843685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:17.848988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b7ed3763c446f684f198ecc096c4eb', 'request_id': 'req-6ddc7efb-5861-40db-a794-ed33c350dc43'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.862746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] The token cache is being invalidate because project 87b7ed3763c446f684f198ecc096c4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:17.863030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.863658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-6ddc7efb-5861-40db-a794-ed33c350dc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:17.865279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1332/2656] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:17 2022] DELETE /identity/v3/projects/87b7ed3763c446f684f198ecc096c4eb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:17.870435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:17.870909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:17.871034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:17.890828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:17.892040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:17.956915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1325/2657] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:17.966278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:17.966583 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:17.966862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:17.986037 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:17.987188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:17.996406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1333/2658] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:18.001400 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.014885 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2c697-2014-4cee-a38a-c23a78414d88 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.020778 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.021493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.021767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.022160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.022980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:18.023794 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.025895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.042546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1326/2659] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:18.046946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.056703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c1d9d-5188-4545-b4f5-5858ba75db9a None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.061598 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.062256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.062537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.062812 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.063568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.065521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.093900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1334/2660] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:18 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:18.098112 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.109684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc4673-34f4-4a11-b92d-2b006f2e64ab admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.169652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.170341 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.170620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.170892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.171580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.173578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.196154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1327/2661] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:18 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:18.200480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.211107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d4683-29e3-48f5-bc4f-d11b48520200 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.216385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.217092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.217366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.217634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.218368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.219963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.230944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1335/2662] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:18.235326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.245989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8bb44-a1b4-4d76-85ad-05d9e6d4ba7c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.250795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921bd22-d51c-4b26-b7b6-74d86c95fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.251461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921bd22-d51c-4b26-b7b6-74d86c95fac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.251744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921bd22-d51c-4b26-b7b6-74d86c95fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.252016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921bd22-d51c-4b26-b7b6-74d86c95fac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.252799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921bd22-d51c-4b26-b7b6-74d86c95fac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.254429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921bd22-d51c-4b26-b7b6-74d86c95fac1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.264865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1328/2663] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:18.269025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.280397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05471ea-ccc1-4322-b942-b9168f0ea102 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.285484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.286358 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.286629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.286906 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] PATH_INFO: `/v3/projects/4f24fa63081d4e4284b22da98d2f7f29/users/92337b817f6c43e3bc96abf0d6cb3f06/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.303266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f24fa63081d4e4284b22da98d2f7f29, user_id=92337b817f6c43e3bc96abf0d6cb3f06, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.306110 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.320556 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1336/2664] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:18 2022] PUT /identity/v3/projects/4f24fa63081d4e4284b22da98d2f7f29/users/92337b817f6c43e3bc96abf0d6cb3f06/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:18.330946 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.331328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.331694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.359819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] MFA Rules not processed for user `92337b817f6c43e3bc96abf0d6cb3f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:18.361122 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:18.396921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:18.432917 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1329/2665] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:18.437541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.503341 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0b43e-12fb-44b0-8e6e-c106c07aa3b4 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.513874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.514552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.514837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.515107 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.519693 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.521672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.562535 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1337/2666] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:18 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:18.569029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.580975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1fe7f-05f4-41d4-b73e-56687ea683f6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.642744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.643428 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.643698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.643980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.644699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.646661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.673217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1330/2667] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:18 2022] POST /identity/v3/users => generated 301 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:18.677275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.688946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d864b-28a1-4ef7-b2e6-41c8117534b2 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.694396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.695155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.695575 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.695845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.696566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.698282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.709486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1338/2668] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:18.714179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.728566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bf25-23b3-4f71-b28f-2a0f2ddd1608 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.734609 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.735497 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.735782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.736052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] PATH_INFO: `/v3/projects/87e0ca7b20f642ee92faf6fd8f4d2259/users/bf3f267cb7674b54885e7c2f690ab661/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.752170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e0ca7b20f642ee92faf6fd8f4d2259, user_id=bf3f267cb7674b54885e7c2f690ab661, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.754956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.769857 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1331/2669] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:18 2022] PUT /identity/v3/projects/87e0ca7b20f642ee92faf6fd8f4d2259/users/bf3f267cb7674b54885e7c2f690ab661/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:18.774508 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.785388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96fad0-da20-46ce-85a3-e937ebdd0c1a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.839074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:18.848790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f82b5-197a-4c79-9a01-fc4707be5b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.848790 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f82b5-197a-4c79-9a01-fc4707be5b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.849177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f82b5-197a-4c79-9a01-fc4707be5b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.849443 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f82b5-197a-4c79-9a01-fc4707be5b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.851124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f82b5-197a-4c79-9a01-fc4707be5b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.853648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f82b5-197a-4c79-9a01-fc4707be5b16 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.864532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1339/2670] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:18.869879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:18.881463 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8e8d1-343f-4cfb-ba7f-8f2e7bd62e28 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:18.888035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:18.888992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.889283 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.889534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] PATH_INFO: `/v3/projects/87e0ca7b20f642ee92faf6fd8f4d2259/users/bf3f267cb7674b54885e7c2f690ab661/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.891835 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e0ca7b20f642ee92faf6fd8f4d2259, user_id=bf3f267cb7674b54885e7c2f690ab661, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:18.894527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:18.906657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1332/2671] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:18 2022] PUT /identity/v3/projects/87e0ca7b20f642ee92faf6fd8f4d2259/users/bf3f267cb7674b54885e7c2f690ab661/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:18.917560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:18.917850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:18.918129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:18.946780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] MFA Rules not processed for user `bf3f267cb7674b54885e7c2f690ab661`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:18.948017 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:19.033492 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1340/2672] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:19.045455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:19.047417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:19.108166 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e019ad3-0210-4823-88e5-ed03fc725c0e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:19.112975 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f41d64-a258-4620-b9ca-d151b539494d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:19.113652 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f41d64-a258-4620-b9ca-d151b539494d service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:19.113935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f41d64-a258-4620-b9ca-d151b539494d service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:19.114206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f41d64-a258-4620-b9ca-d151b539494d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:19.124547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f41d64-a258-4620-b9ca-d151b539494d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:19.126022 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f41d64-a258-4620-b9ca-d151b539494d service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:19.135476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1333/2673] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:19.280562 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:19.350955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917be4e0-ca19-4497-9925-36bf0c53a37e None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:19.357485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:19.358374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:19.358651 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:19.359078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] PATH_INFO: `/v3/users/92337b817f6c43e3bc96abf0d6cb3f06` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:19.360420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=92337b817f6c43e3bc96abf0d6cb3f06)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:19.362390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:19.420415 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92337b817f6c43e3bc96abf0d6cb3f06'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:19.421129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92337b817f6c43e3bc96abf0d6cb3f06'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:19.433207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92337b817f6c43e3bc96abf0d6cb3f06'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:19.433892 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92337b817f6c43e3bc96abf0d6cb3f06'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:19.441487 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92337b817f6c43e3bc96abf0d6cb3f06'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:19.447336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:19.455615 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1341/2674] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:19 2022] DELETE /identity/v3/users/92337b817f6c43e3bc96abf0d6cb3f06 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:19.460418 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:19.460732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:19.461049 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:19.480123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:19.481362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:19.505995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:19.546928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1334/2675] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:19.556695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:19.558185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:19.615338 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20394e8-8f7f-4354-881c-c2303ba60ffc admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:19.619712 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:19.620365 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:19.620737 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:19.620905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:19.630317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:19.631762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:19.641489 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1342/2676] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:19.918679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:19.920361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:19.935033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bfbad-2d87-4b5d-9a31-a45837813c19 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:19.940828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:19.941795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:19.942043 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:19.942319 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] PATH_INFO: `/v3/projects/4f24fa63081d4e4284b22da98d2f7f29` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:19.943558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f24fa63081d4e4284b22da98d2f7f29)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:19.945568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:19.993150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f24fa63081d4e4284b22da98d2f7f29', 'request_id': 'req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.001378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f24fa63081d4e4284b22da98d2f7f29', 'request_id': 'req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.007371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f24fa63081d4e4284b22da98d2f7f29', 'request_id': 'req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.019126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] The token cache is being invalidate because project 4f24fa63081d4e4284b22da98d2f7f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:20.019415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.020063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.021695 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1335/2677] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:19 2022] DELETE /identity/v3/projects/4f24fa63081d4e4284b22da98d2f7f29 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:20.026406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:20.037247 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca3b90-e250-4b9b-b5e0-8188b77bb849 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:20.094869 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:20.095750 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.096029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.096303 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] PATH_INFO: `/v3/users/bf3f267cb7674b54885e7c2f690ab661` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.097558 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3f267cb7674b54885e7c2f690ab661)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:20.099479 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:20.147718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3f267cb7674b54885e7c2f690ab661'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.148765 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3f267cb7674b54885e7c2f690ab661'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.161205 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3f267cb7674b54885e7c2f690ab661'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.161847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3f267cb7674b54885e7c2f690ab661'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.171200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-955a5fcc-3f01-4d78-bae0-bdbe599f1582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3f267cb7674b54885e7c2f690ab661'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.188234 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1343/2678] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:20 2022] DELETE /identity/v3/users/bf3f267cb7674b54885e7c2f690ab661 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:20.473197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:20.475429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:20.491679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614e624-5ba7-40b0-a72d-b63b59ca4b1f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:20.572083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:20.573236 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.573563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.573983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] PATH_INFO: `/v3/projects/87e0ca7b20f642ee92faf6fd8f4d2259` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.575352 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e0ca7b20f642ee92faf6fd8f4d2259)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:20.577759 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:20.628340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e0ca7b20f642ee92faf6fd8f4d2259', 'request_id': 'req-ac031e06-45c5-48d5-b50c-0fcef43e2999'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.632029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:20.636768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e0ca7b20f642ee92faf6fd8f4d2259', 'request_id': 'req-ac031e06-45c5-48d5-b50c-0fcef43e2999'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.643932 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e0ca7b20f642ee92faf6fd8f4d2259', 'request_id': 'req-ac031e06-45c5-48d5-b50c-0fcef43e2999'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.657026 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] The token cache is being invalidate because project 87e0ca7b20f642ee92faf6fd8f4d2259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:20.657316 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.657980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ac031e06-45c5-48d5-b50c-0fcef43e2999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:20.660082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1336/2679] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:20 2022] DELETE /identity/v3/projects/87e0ca7b20f642ee92faf6fd8f4d2259 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:20.665276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.665781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.665781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.683330 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:20.684540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:20.697348 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:20.752177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1344/2680] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:20.762774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.763059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.763332 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.785216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:20.786421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:20.801376 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1337/2681] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:20.806316 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:20.817248 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb193e0-4f3e-4b21-b6c2-7e6c20325f90 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:20.824597 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:20.825435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.825717 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.825992 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.826755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:20.827065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:20.828732 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:20.846794 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1345/2682] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:20.852774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:20.862787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2355a6-7c53-4d0a-b6d1-28bb0a149510 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:20.868915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:20.869583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.869860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.870129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.870815 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:20.872743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:20.901401 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1338/2683] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:20 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:20.906008 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:20.919722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613502e0-513e-4d68-b4f2-f20f735f001f admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:20.979626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:20.980298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:20.980577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:20.980893 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:20.981587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:20.983514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:21.005541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1346/2684] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:20 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:21.009783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:21.020314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa25c20-dcce-4c88-889b-d3a2b6e27f6c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:21.025573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:21.026229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:21.026506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:21.026777 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:21.027501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:21.029149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:21.040435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1339/2685] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:21.044377 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:21.055092 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc4d8-a466-4390-bf41-412a4f9ecc81 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:21.060667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e9531-5928-4b17-b3fb-ee7c45a6189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:21.061375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9531-5928-4b17-b3fb-ee7c45a6189b admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:21.061653 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9531-5928-4b17-b3fb-ee7c45a6189b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:21.061929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e9531-5928-4b17-b3fb-ee7c45a6189b admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:21.062667 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9531-5928-4b17-b3fb-ee7c45a6189b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:21.064273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e9531-5928-4b17-b3fb-ee7c45a6189b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:21.074592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1347/2686] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:21.078688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:21.088864 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b288c-eb6c-4b28-a624-1912a9dd2c73 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:21.093848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:21.094754 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:21.095033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:21.095308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] PATH_INFO: `/v3/projects/df6d2a5aa6d04dc4a372d8ca4899f4ef/users/06c71d8f1be545629835db4f2a25f20a/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:21.112059 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6d2a5aa6d04dc4a372d8ca4899f4ef, user_id=06c71d8f1be545629835db4f2a25f20a, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:21.114786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:21.128561 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1340/2687] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:21 2022] PUT /identity/v3/projects/df6d2a5aa6d04dc4a372d8ca4899f4ef/users/06c71d8f1be545629835db4f2a25f20a/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:21.139274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:21.139567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:21.139855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:21.166604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] MFA Rules not processed for user `06c71d8f1be545629835db4f2a25f20a`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:21.167845 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:21.237519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1348/2688] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:21.249304 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:21.251519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:21.314586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809441f-93fc-4e89-88b7-b9928221b802 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:21.318756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fe5ce-e169-40f9-93e8-db7c04954f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:21.319466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fe5ce-e169-40f9-93e8-db7c04954f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:21.319744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fe5ce-e169-40f9-93e8-db7c04954f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:21.320017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fe5ce-e169-40f9-93e8-db7c04954f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:21.328806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fe5ce-e169-40f9-93e8-db7c04954f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:21.330254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fe5ce-e169-40f9-93e8-db7c04954f8a service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:21.340143 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1341/2689] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:28.795959 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:28.810183 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:28.860259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ad41f-aa00-4d4d-8175-bcb3ae8746c9 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:28.865439 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:28.866312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:28.866592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:28.866862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] PATH_INFO: `/v3/users/06c71d8f1be545629835db4f2a25f20a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:28.868068 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c71d8f1be545629835db4f2a25f20a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:28.870011 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:28.921802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c71d8f1be545629835db4f2a25f20a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:28.922408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c71d8f1be545629835db4f2a25f20a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:28.934567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c71d8f1be545629835db4f2a25f20a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:28.935235 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c71d8f1be545629835db4f2a25f20a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:28.944408 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c71d8f1be545629835db4f2a25f20a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:28.957227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1349/2690] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:28 2022] DELETE /identity/v3/users/06c71d8f1be545629835db4f2a25f20a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:28.961821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:28.962324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:28.962431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:28.979274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:28.980478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:29.060238 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1342/2691] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:29.070755 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.073298 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:29.136047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef66e-4ff1-4905-b342-0ed45fde0f77 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.140266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.140949 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.141245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.141496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.151148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.152594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.162291 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1350/2692] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:29.470683 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.472277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:29.474722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.484858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04045cb5-604f-46e9-8fe4-4901d9b27c88 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.489655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.490525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.490804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.491073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] PATH_INFO: `/v3/projects/df6d2a5aa6d04dc4a372d8ca4899f4ef` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.491988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367aa80-0df8-4422-ae40-c78cc75d95c1 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.492261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6d2a5aa6d04dc4a372d8ca4899f4ef)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.494216 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.495760 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.496299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.496724 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.497015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.500078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:29.513909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:29.514087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.515481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d9b65-23af-437a-8bd4-a50ea973d739 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.526351 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1351/2693] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:29.547156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6d2a5aa6d04dc4a372d8ca4899f4ef', 'request_id': 'req-b03a9556-920d-4295-a808-0b13c707e9a3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:29.555164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6d2a5aa6d04dc4a372d8ca4899f4ef', 'request_id': 'req-b03a9556-920d-4295-a808-0b13c707e9a3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:29.561179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6d2a5aa6d04dc4a372d8ca4899f4ef', 'request_id': 'req-b03a9556-920d-4295-a808-0b13c707e9a3'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:29.572874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] The token cache is being invalidate because project df6d2a5aa6d04dc4a372d8ca4899f4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:29.573161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:29.573950 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b03a9556-920d-4295-a808-0b13c707e9a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:29.575826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1343/2694] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:29 2022] DELETE /identity/v3/projects/df6d2a5aa6d04dc4a372d8ca4899f4ef => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:29.580762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.581050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.581322 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.599681 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:29.600943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:29.618555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:29.666744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1352/2695] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:29.675929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.676209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.676491 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.695450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:29.696667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:29.705784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1344/2696] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:29.710166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.720172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0e9b2-9200-4577-8c71-70bab91cb30f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.725173 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.725846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.726128 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.726437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.727162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:29.727468 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.729102 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.741375 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1353/2697] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:29.749312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.758983 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c569f6e-1a24-4c9c-b389-68231a016294 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.763891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.764578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.764888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.765160 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.765852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.767810 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.795123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1345/2698] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:29.800097 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.810762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec953cd-22d8-46e5-988e-8c81cb5d53b8 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.868493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.869219 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.869532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.869945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.870655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.872741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.893973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1354/2699] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:29 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:29.897784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.909178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56fad3-0903-41fd-b498-587c9a517d24 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.914581 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.915274 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.915558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.915832 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.916563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.918207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.929738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1346/2700] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:29.933253 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.943821 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960059cc-bbba-4131-833a-0e8af9419045 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.949282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87e8b6-e0ce-42b9-8017-2de7c260ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.950176 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e8b6-e0ce-42b9-8017-2de7c260ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.950511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e8b6-e0ce-42b9-8017-2de7c260ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.950847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e8b6-e0ce-42b9-8017-2de7c260ad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:29.951679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87e8b6-e0ce-42b9-8017-2de7c260ad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:29.953410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87e8b6-e0ce-42b9-8017-2de7c260ad97 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:29.964509 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1355/2701] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:29.968259 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:29.980544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89714619-0877-41dc-aa31-ed1844c273cf admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:29.985785 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:29.986742 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:29.987024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:29.987320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] PATH_INFO: `/v3/projects/e063fa76f20e48b8a1bea39f65dbb513/users/6ef1ace7981240adb5e2e48472cbf711/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.002701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e063fa76f20e48b8a1bea39f65dbb513, user_id=6ef1ace7981240adb5e2e48472cbf711, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.005403 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.020241 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1347/2702] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:29 2022] PUT /identity/v3/projects/e063fa76f20e48b8a1bea39f65dbb513/users/6ef1ace7981240adb5e2e48472cbf711/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:30.087616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.087911 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.088186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.118584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] MFA Rules not processed for user `6ef1ace7981240adb5e2e48472cbf711`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:30.119852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:30.197203 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1356/2703] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:30.201858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.265156 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1b92-ac95-4cce-91ec-31040a7f3f0c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.270504 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.271203 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.271485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.271753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.272476 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.274802 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.302577 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1348/2704] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:30 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:30.306728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.318943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df9f0-1ccb-4915-920f-93adb3386648 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.331882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:30.379416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.380104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.380399 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.380706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.381410 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.383416 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.411623 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1357/2705] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:30 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:30.415672 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.429442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839b7d9-eb78-43b8-b1a6-59a809cdf0d7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.435453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.436151 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.436429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.436753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.437598 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.439224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.454065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1349/2706] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:30.458698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.470009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231845d6-b6a9-460f-a5b6-bbf64a428e8c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.475053 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8c072-85ef-47bd-9b6e-d014f9b1e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.476059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8c072-85ef-47bd-9b6e-d014f9b1e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.477166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8c072-85ef-47bd-9b6e-d014f9b1e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.477457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8c072-85ef-47bd-9b6e-d014f9b1e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.478343 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8c072-85ef-47bd-9b6e-d014f9b1e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.480061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8c072-85ef-47bd-9b6e-d014f9b1e6aa admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.495150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1358/2707] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:30.499806 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.510535 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e052-5ec4-41c9-a9a5-3fcd33623229 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.517620 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.518534 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.518819 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.519092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] PATH_INFO: `/v3/projects/bdb61d2c0d744c9cac575aca41d9234b/users/22764f895a714d05802657d04d336c6d/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.534417 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:30.537079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb61d2c0d744c9cac575aca41d9234b, user_id=22764f895a714d05802657d04d336c6d, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.539914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.557739 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1350/2708] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:30 2022] PUT /identity/v3/projects/bdb61d2c0d744c9cac575aca41d9234b/users/22764f895a714d05802657d04d336c6d/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:30.568339 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.568620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.568919 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.609715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] MFA Rules not processed for user `22764f895a714d05802657d04d336c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:30.611015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:30.681294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1359/2709] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:30.685893 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.749694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d1c5f-86b7-4a5d-b043-d658ee117756 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.754495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.755162 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.755441 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.755717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.756442 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.758378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.787281 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1351/2710] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:30 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:30.791296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.802284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8d458-3c24-4a01-86de-e96f63f54cee None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.861976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.862695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.862983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.863273 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.863976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.865995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.891337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1360/2711] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:30 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:30.892419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.908037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe085d6-0aac-41d6-bc63-e1b958d2be04 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.913782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.914479 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.914761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.915031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.915772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.917517 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.929097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1352/2712] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:30.933547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:30.944226 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a8e7-649c-43fd-93a9-bef14f57d804 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:30.949044 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:30.949983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:30.950265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:30.950539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] PATH_INFO: `/v3/projects/3dac2bdc24d6463486025dd734eaadd0/users/818841f0b8d341b98f82082c3f2529eb/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:30.963998 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:30.967626 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dac2bdc24d6463486025dd734eaadd0, user_id=818841f0b8d341b98f82082c3f2529eb, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:30.971057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:30.988933 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1361/2713] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:30 2022] PUT /identity/v3/projects/3dac2bdc24d6463486025dd734eaadd0/users/818841f0b8d341b98f82082c3f2529eb/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:30.993016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:31.003763 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c53c-0761-4ad7-ae0b-7aae8be867be admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:31.067080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ada09-ba32-4332-86a8-5503e3c45fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:31.067764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ada09-ba32-4332-86a8-5503e3c45fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:31.068047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ada09-ba32-4332-86a8-5503e3c45fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:31.068321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ada09-ba32-4332-86a8-5503e3c45fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:31.069212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ada09-ba32-4332-86a8-5503e3c45fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:31.070842 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ada09-ba32-4332-86a8-5503e3c45fc8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:31.083870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1353/2714] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:31.088518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:31.100295 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea175e-5bc4-4fd0-b613-54b6ffd6e2d6 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:31.105420 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:31.106396 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:31.106655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:31.107545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] PATH_INFO: `/v3/projects/3dac2bdc24d6463486025dd734eaadd0/users/818841f0b8d341b98f82082c3f2529eb/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:31.110709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dac2bdc24d6463486025dd734eaadd0, user_id=818841f0b8d341b98f82082c3f2529eb, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:31.113627 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:31.125850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1362/2715] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:31 2022] PUT /identity/v3/projects/3dac2bdc24d6463486025dd734eaadd0/users/818841f0b8d341b98f82082c3f2529eb/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:31.137213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:31.137558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:31.137858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:31.174419 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] MFA Rules not processed for user `818841f0b8d341b98f82082c3f2529eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:31.175750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:31.226386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:31.251915 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1354/2716] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:31.263463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:31.265725 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:31.324953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94fa2f-061e-4a65-b356-3a3c3f87f018 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:31.329009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:31.329682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:31.330025 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:31.330233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:31.397696 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:31.399548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:31.410418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1363/2717] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:32.521176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:32.522676 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:32.597909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f31fb5-776f-4c99-a9aa-6384da3e5392 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:32.603326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:32.604207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:32.604499 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:32.604809 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] PATH_INFO: `/v3/users/7294e5eafdfc42fbac105cf58ee07b70` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:32.606058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7294e5eafdfc42fbac105cf58ee07b70)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:32.607998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:32.657408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7294e5eafdfc42fbac105cf58ee07b70'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:32.658092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7294e5eafdfc42fbac105cf58ee07b70'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:32.670969 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7294e5eafdfc42fbac105cf58ee07b70'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:32.671622 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7294e5eafdfc42fbac105cf58ee07b70'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:32.680927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7294e5eafdfc42fbac105cf58ee07b70'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:32.698952 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1355/2718] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:32 2022] DELETE /identity/v3/users/7294e5eafdfc42fbac105cf58ee07b70 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:32.883709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:32.896472 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4ba5-9be4-4c58-ad34-3f46fc890b73 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:32.942220 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:32.943953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:32.957486 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:32.958480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:32.958781 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:32.959032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] PATH_INFO: `/v3/projects/373556d3416e4db68d08897c3d7c0a54` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:32.960212 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=373556d3416e4db68d08897c3d7c0a54)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:32.962137 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.002184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:33.004546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4378-ef68-440c-9e48-002a15b8e0f8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.015272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373556d3416e4db68d08897c3d7c0a54', 'request_id': 'req-acc4647b-18f6-490f-bf60-8d77a8bd1276'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:33.023826 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:33.024202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373556d3416e4db68d08897c3d7c0a54', 'request_id': 'req-acc4647b-18f6-490f-bf60-8d77a8bd1276'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:33.031043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373556d3416e4db68d08897c3d7c0a54', 'request_id': 'req-acc4647b-18f6-490f-bf60-8d77a8bd1276'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:33.050855 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] The token cache is being invalidate because project 373556d3416e4db68d08897c3d7c0a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:33.051112 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:33.051757 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-acc4647b-18f6-490f-bf60-8d77a8bd1276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:33.053424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1364/2719] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:32 2022] DELETE /identity/v3/projects/373556d3416e4db68d08897c3d7c0a54 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:33.058411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.058686 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.058962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.066226 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.066910 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.067187 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.067458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.069323 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:33.082274 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:33.083476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:33.135868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.137308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfa485-e70b-4878-8757-b7823ec54fd4 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.146464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1356/2720] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:33.149220 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1365/2721] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:33.156998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.157288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.157568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.176153 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:33.176995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:33.185416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1357/2722] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:33.189848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.199647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b46aa3-9f5b-46b6-b188-396503e40806 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.204481 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.205262 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.205716 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.206048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.206851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:33.207230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.208912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.222284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1366/2723] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:33.226776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.237252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b199b-3dd6-4280-8a62-f5e0ae00ab90 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.244973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.246036 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.246565 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.247097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.248361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.251727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.284586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1358/2724] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:33.288843 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.299564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f8ab3-0421-4669-9c3c-6fa9e0c3e2f2 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.357345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.358340 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.358752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.359126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.360139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.362836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.385735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1367/2725] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/users => generated 280 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:33.390258 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.401774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c1cd0-2699-4483-829f-16a95a689d4a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.406783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.407458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.407741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.408015 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.408782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.410429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.424072 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1359/2726] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:33.425319 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.435621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22d007-f13d-47a8-b6fa-62d79f2dfb3c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.440804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c495d-2c06-40d3-a779-93762bb2135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.441454 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c495d-2c06-40d3-a779-93762bb2135f admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.441736 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c495d-2c06-40d3-a779-93762bb2135f admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.442009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c495d-2c06-40d3-a779-93762bb2135f admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.442743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c495d-2c06-40d3-a779-93762bb2135f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.444356 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c495d-2c06-40d3-a779-93762bb2135f admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.458661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1368/2727] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:33.460211 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.471431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702360-7ab7-407d-89f0-4d5cadd6b19e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.476118 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.477101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.477383 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.477651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] PATH_INFO: `/v3/projects/e3b2af5a4c19424b9d97192bac1277d2/users/77bab8203aa649b0b534e60eb70cc3c5/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.495016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2af5a4c19424b9d97192bac1277d2, user_id=77bab8203aa649b0b534e60eb70cc3c5, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.497798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.512870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1360/2728] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:33 2022] PUT /identity/v3/projects/e3b2af5a4c19424b9d97192bac1277d2/users/77bab8203aa649b0b534e60eb70cc3c5/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:33.522661 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.522946 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.523221 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.553010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] MFA Rules not processed for user `77bab8203aa649b0b534e60eb70cc3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:33.554202 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:33.572936 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:33.627917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1369/2729] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:33.632178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.695764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616a85c-af4d-48fb-a772-a85aaeeb5c7a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.701101 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.701780 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.702096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.702369 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.703287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.704147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.705250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.705697 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:33.726114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:33.738702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1361/2730] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:33.739629 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.767741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22cc67-6c01-43dc-9ee6-d18cec2b58f8 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.802602 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f88a-bc83-441e-b004-daa7b631f77e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.807296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.807981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.808263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.808540 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.809438 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.811413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.817059 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.817709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.818155 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.818425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.820711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:33.831008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1362/2731] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:33 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:33.834993 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.844718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444c16-9733-4a71-b565-e34b3e2b3e0f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.849390 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.850047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.850333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.850610 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.851334 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.852965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.864972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1363/2732] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:33.869078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.879087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9d6a2-0d1b-42ad-afb5-8f7b00aada55 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.883603 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:33.884459 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:33.884766 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:33.885040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] PATH_INFO: `/v3/projects/716c5b46c85c4d0eb0afe8c2d6dfa5fc/users/f517c461a76847d0b80809ba320c1607/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:33.885210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.886860 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.896483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1370/2733] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:44:33.900833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=716c5b46c85c4d0eb0afe8c2d6dfa5fc, user_id=f517c461a76847d0b80809ba320c1607, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:33.904189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:33.921396 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1364/2734] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:33 2022] PUT /identity/v3/projects/716c5b46c85c4d0eb0afe8c2d6dfa5fc/users/f517c461a76847d0b80809ba320c1607/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:33.925573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:33.936764 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb024aa-89a5-42f7-b65b-c1e7c9bb6651 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:33.957577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=100 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:34.000568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee57f9-ad1b-48e0-b31f-a3be52be2b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:34.001283 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee57f9-ad1b-48e0-b31f-a3be52be2b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:34.001564 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee57f9-ad1b-48e0-b31f-a3be52be2b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:34.001837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee57f9-ad1b-48e0-b31f-a3be52be2b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:34.002570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee57f9-ad1b-48e0-b31f-a3be52be2b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:34.004188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee57f9-ad1b-48e0-b31f-a3be52be2b48 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:34.014482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1371/2735] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:34.018698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:34.029987 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cebd5-9fc1-4729-8437-9e5e4e9ab023 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:34.034597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:34.035506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:34.035789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:34.036058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] PATH_INFO: `/v3/projects/716c5b46c85c4d0eb0afe8c2d6dfa5fc/users/f517c461a76847d0b80809ba320c1607/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:34.038035 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=716c5b46c85c4d0eb0afe8c2d6dfa5fc, user_id=f517c461a76847d0b80809ba320c1607, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:34.040724 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:34.051708 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1365/2736] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:34 2022] PUT /identity/v3/projects/716c5b46c85c4d0eb0afe8c2d6dfa5fc/users/f517c461a76847d0b80809ba320c1607/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:34.063072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:34.063360 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:34.063636 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:34.097927 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] MFA Rules not processed for user `f517c461a76847d0b80809ba320c1607`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:34.099146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:34.174175 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1372/2737] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:34.185908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:34.248206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588adff-4d06-48d1-bea7-a416b6e356f4 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:34.252868 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e096e-337f-40dc-a326-ceda9712ea46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:34.253585 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e096e-337f-40dc-a326-ceda9712ea46 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:34.253942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e096e-337f-40dc-a326-ceda9712ea46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:34.254251 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e096e-337f-40dc-a326-ceda9712ea46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:34.265031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e096e-337f-40dc-a326-ceda9712ea46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:34.266464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e096e-337f-40dc-a326-ceda9712ea46 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:34.276389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1366/2738] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:34.701654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:34.779544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2148f-068e-4266-ba51-e67547faddd8 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:34.784227 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:34.785146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:34.785429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:34.785700 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] PATH_INFO: `/v3/users/77bab8203aa649b0b534e60eb70cc3c5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:34.786923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] RBAC: Authorizing `identity:delete_user(user_id=77bab8203aa649b0b534e60eb70cc3c5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:34.788887 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:34.837390 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77bab8203aa649b0b534e60eb70cc3c5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:34.838050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77bab8203aa649b0b534e60eb70cc3c5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:34.839167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:34.851604 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77bab8203aa649b0b534e60eb70cc3c5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:34.852311 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77bab8203aa649b0b534e60eb70cc3c5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:34.861003 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77bab8203aa649b0b534e60eb70cc3c5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:34.875521 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1373/2739] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:34 2022] DELETE /identity/v3/users/77bab8203aa649b0b534e60eb70cc3c5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:34.880287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:34.880821 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:34.880922 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:34.901295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:34.902519 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:34.941168 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:34.973847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1367/2740] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:34.985275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:34.986961 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:35.062554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b66b97-1cdc-4de9-b222-f5d07623405e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:35.067144 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:35.067803 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:35.072485 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:35.072797 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:35.087584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:35.089587 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:35.101570 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1374/2741] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:35.418816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:35.420444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:35.432404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166c50b-6a62-402d-bbab-48b595f5f3eb None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:35.437412 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:35.438359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:35.438701 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:35.439034 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] PATH_INFO: `/v3/projects/e3b2af5a4c19424b9d97192bac1277d2` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:35.440317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b2af5a4c19424b9d97192bac1277d2)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:35.442282 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:35.492066 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b2af5a4c19424b9d97192bac1277d2', 'request_id': 'req-eb4c49c8-1229-4e08-8694-ec88740ca44c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.501230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b2af5a4c19424b9d97192bac1277d2', 'request_id': 'req-eb4c49c8-1229-4e08-8694-ec88740ca44c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.507393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b2af5a4c19424b9d97192bac1277d2', 'request_id': 'req-eb4c49c8-1229-4e08-8694-ec88740ca44c'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.518888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] The token cache is being invalidate because project e3b2af5a4c19424b9d97192bac1277d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:35.519176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.519904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.521612 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1368/2742] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:35 2022] DELETE /identity/v3/projects/e3b2af5a4c19424b9d97192bac1277d2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:35.526165 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:35.536268 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a03d4-7304-4c4c-ae75-fdd199ae1afa service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:35.595955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:35.596882 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:35.597163 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:35.597435 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] PATH_INFO: `/v3/users/f517c461a76847d0b80809ba320c1607` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:35.598668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f517c461a76847d0b80809ba320c1607)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:35.600654 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:35.650947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f517c461a76847d0b80809ba320c1607'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.651669 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f517c461a76847d0b80809ba320c1607'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.665028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f517c461a76847d0b80809ba320c1607'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.665685 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f517c461a76847d0b80809ba320c1607'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.669972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:35.674580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-26ac6dde-ba5a-4324-80ef-d1b5dbc307cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f517c461a76847d0b80809ba320c1607'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:35.689164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1375/2743] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:35 2022] DELETE /identity/v3/users/f517c461a76847d0b80809ba320c1607 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:35.969792 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:35.971688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:35.986052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c49c8-1229-4e08-8694-ec88740ca44c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.053398 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.054197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.054478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.054752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] PATH_INFO: `/v3/projects/716c5b46c85c4d0eb0afe8c2d6dfa5fc` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.056028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=716c5b46c85c4d0eb0afe8c2d6dfa5fc)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.058006 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.107597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '716c5b46c85c4d0eb0afe8c2d6dfa5fc', 'request_id': 'req-b888f0a6-9c49-4e45-8026-59093d8bbff0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:36.117312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '716c5b46c85c4d0eb0afe8c2d6dfa5fc', 'request_id': 'req-b888f0a6-9c49-4e45-8026-59093d8bbff0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:36.123214 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '716c5b46c85c4d0eb0afe8c2d6dfa5fc', 'request_id': 'req-b888f0a6-9c49-4e45-8026-59093d8bbff0'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:36.125109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:36.135763 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] The token cache is being invalidate because project 716c5b46c85c4d0eb0afe8c2d6dfa5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:36.136228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:36.136964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-b888f0a6-9c49-4e45-8026-59093d8bbff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:36.138914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1369/2744] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:35 2022] DELETE /identity/v3/projects/716c5b46c85c4d0eb0afe8c2d6dfa5fc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:36.144208 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.144701 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.144802 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.163568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:36.164832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:36.228015 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1376/2745] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:36.237208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.237543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.237839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.258433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:36.259654 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:36.269432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1370/2746] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:36.276350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.287362 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d07f8-47ba-4b1b-88e9-323a3021c51b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.293010 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.293789 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.294282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.294282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.294993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:36.295327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.297074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.309664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1377/2747] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:36.314696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.324905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68236b-d317-44a1-8f15-e984bd8b0f14 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.331114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.331848 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.332194 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.332530 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.333329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.335484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.373727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1371/2748] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:36.375254 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.385574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2e643-d8ce-47e5-8876-0e14eebd7ca7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.454608 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.455294 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.455585 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.455828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.456510 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.458512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.490122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1378/2749] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:36.494129 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.505433 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa673b-65cc-42ac-92d9-549dce355ea9 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.510113 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.510782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.511060 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.511328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.512180 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.514068 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.529351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1372/2750] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:36.530143 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.540418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7307f9-33f8-4925-a02a-147b9a62dc91 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.545633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40252886-eb6c-4601-9b45-046bc5816c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.546299 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40252886-eb6c-4601-9b45-046bc5816c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.546577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40252886-eb6c-4601-9b45-046bc5816c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.546850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40252886-eb6c-4601-9b45-046bc5816c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.547581 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40252886-eb6c-4601-9b45-046bc5816c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.549230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40252886-eb6c-4601-9b45-046bc5816c25 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.559476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1379/2751] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:36.563947 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.578307 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81303b-febe-418f-8695-081d4c93a0a8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.583195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.584094 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.584371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.584667 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] PATH_INFO: `/v3/projects/9b35edebd9434f76a82d6281e0b4e631/users/d190877b4df34f3292d692199efe66d5/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.600228 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b35edebd9434f76a82d6281e0b4e631, user_id=d190877b4df34f3292d692199efe66d5, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.602937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.617936 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1373/2752] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:36 2022] PUT /identity/v3/projects/9b35edebd9434f76a82d6281e0b4e631/users/d190877b4df34f3292d692199efe66d5/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:36.629706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.630994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.630994 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.660706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] MFA Rules not processed for user `d190877b4df34f3292d692199efe66d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:36.662161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:36.710476 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:36.734308 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1380/2753] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:36.738938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.833782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993dae7-aa44-4ede-9ceb-a06435bde545 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.840016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.840891 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.841179 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.841483 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.842132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.844058 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.875502 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1374/2754] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:36.879752 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.891502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c1bd-6a0b-4481-bc96-f4ca6b907480 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:36.955759 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:36.956450 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:36.956810 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:36.957091 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:36.957784 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:36.959739 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:36.979734 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1381/2755] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:36 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:36.983851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:36.995069 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef374073-5a37-488b-a7dd-36d0d43ce9e5 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:37.000787 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:37.001458 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.001733 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.001999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.002727 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:37.004331 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:37.015532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1375/2756] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:37.019947 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:37.031167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8e421-54f8-4e3d-b11d-40629485d5c0 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:37.036645 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:37.037567 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.037854 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.038151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] PATH_INFO: `/v3/projects/73e88ee5790c48b7a91c2fef947d2f9a/users/fba49625bd05432f972631cea935b62a/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.054085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e88ee5790c48b7a91c2fef947d2f9a, user_id=fba49625bd05432f972631cea935b62a, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:37.056816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:37.071168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1382/2757] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:37 2022] PUT /identity/v3/projects/73e88ee5790c48b7a91c2fef947d2f9a/users/fba49625bd05432f972631cea935b62a/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:37.075197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:37.085764 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1b67-f612-4874-9dd4-53a49bcffea1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:37.162142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c057f-f2cb-44ad-acb1-7d502c1aaad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:37.163065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c057f-f2cb-44ad-acb1-7d502c1aaad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.163486 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c057f-f2cb-44ad-acb1-7d502c1aaad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.163899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c057f-f2cb-44ad-acb1-7d502c1aaad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.165717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c057f-f2cb-44ad-acb1-7d502c1aaad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:37.167918 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c057f-f2cb-44ad-acb1-7d502c1aaad0 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:37.180414 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1376/2758] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:37.185596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:37.197177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade3e21-db89-4567-a6c5-d88c6829e3e6 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:37.217124 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:37.218030 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.218309 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.218582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] PATH_INFO: `/v3/projects/73e88ee5790c48b7a91c2fef947d2f9a/users/fba49625bd05432f972631cea935b62a/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.220744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e88ee5790c48b7a91c2fef947d2f9a, user_id=fba49625bd05432f972631cea935b62a, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:37.223579 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:37.235704 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1383/2759] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:37 2022] PUT /identity/v3/projects/73e88ee5790c48b7a91c2fef947d2f9a/users/fba49625bd05432f972631cea935b62a/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:37.247157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.247605 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.248056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.258626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:37.282391 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] MFA Rules not processed for user `fba49625bd05432f972631cea935b62a`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:37.283607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:37.353271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1377/2760] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:37 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:37.365129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:37.367242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:37.431398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8cfca-2740-4052-9707-2a7fdd95c86b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:37.435929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e400c7-1ca6-4ddc-8da3-ed348491c42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:37.436576 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e400c7-1ca6-4ddc-8da3-ed348491c42d service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.436881 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e400c7-1ca6-4ddc-8da3-ed348491c42d service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.437151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e400c7-1ca6-4ddc-8da3-ed348491c42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.446631 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e400c7-1ca6-4ddc-8da3-ed348491c42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:37.448065 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e400c7-1ca6-4ddc-8da3-ed348491c42d service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:37.458474 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1384/2761] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:37.782032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:37.783772 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:37.853500 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db084b0-8e0a-4f91-b750-634baf7b7e5c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:37.858208 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:37.859099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.859501 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.859782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] PATH_INFO: `/v3/users/6ef1ace7981240adb5e2e48472cbf711` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.861114 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef1ace7981240adb5e2e48472cbf711)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:37.863021 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:37.912098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef1ace7981240adb5e2e48472cbf711'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:37.912801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef1ace7981240adb5e2e48472cbf711'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:37.925310 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef1ace7981240adb5e2e48472cbf711'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:37.926186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef1ace7981240adb5e2e48472cbf711'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:37.934240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef1ace7981240adb5e2e48472cbf711'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:37.948421 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1378/2762] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:37 2022] DELETE /identity/v3/users/6ef1ace7981240adb5e2e48472cbf711 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:37.953331 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:37.953610 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:37.953942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:37.973973 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:37.975259 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:38.009098 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:38.041768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1385/2763] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:38.052660 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:38.053962 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:38.065786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:38.118119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf37eeb-1946-4e01-8fd6-998647934962 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:38.122674 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:38.123518 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:38.123698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:38.123972 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:38.136359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:38.138084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:38.148253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1379/2764] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:38.448601 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:38.450945 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:38.463937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946e462-0184-4bb5-ae17-30510e202222 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:38.471035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:38.472162 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:38.473239 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:38.473517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] PATH_INFO: `/v3/projects/e063fa76f20e48b8a1bea39f65dbb513` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:38.474741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e063fa76f20e48b8a1bea39f65dbb513)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:38.476675 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:38.533290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e063fa76f20e48b8a1bea39f65dbb513', 'request_id': 'req-e8a72020-b7e8-4afe-ad8b-37011161b44e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.539320 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e063fa76f20e48b8a1bea39f65dbb513', 'request_id': 'req-e8a72020-b7e8-4afe-ad8b-37011161b44e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.545711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e063fa76f20e48b8a1bea39f65dbb513', 'request_id': 'req-e8a72020-b7e8-4afe-ad8b-37011161b44e'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.561646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] The token cache is being invalidate because project e063fa76f20e48b8a1bea39f65dbb513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:38.562002 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.562753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.564985 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1386/2765] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:38 2022] DELETE /identity/v3/projects/e063fa76f20e48b8a1bea39f65dbb513 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:38.569032 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:38.582590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf079ef-9194-44ed-9b93-9d5b51f4b7b0 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:38.642752 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:38.643586 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:38.643870 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:38.644190 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] PATH_INFO: `/v3/users/22764f895a714d05802657d04d336c6d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:38.645475 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=22764f895a714d05802657d04d336c6d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:38.647816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:38.714930 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22764f895a714d05802657d04d336c6d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.715624 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22764f895a714d05802657d04d336c6d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.728722 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22764f895a714d05802657d04d336c6d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.729464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22764f895a714d05802657d04d336c6d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.737623 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22764f895a714d05802657d04d336c6d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:38.750659 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1380/2766] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:38 2022] DELETE /identity/v3/users/22764f895a714d05802657d04d336c6d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:38.888976 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:38.890589 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:38.904161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a72020-b7e8-4afe-ad8b-37011161b44e admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:38.964020 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:38.964987 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:38.965256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:38.965552 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] PATH_INFO: `/v3/users/d190877b4df34f3292d692199efe66d5` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:38.966846 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d190877b4df34f3292d692199efe66d5)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:38.969135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:39.026371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd190877b4df34f3292d692199efe66d5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.027040 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd190877b4df34f3292d692199efe66d5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.041161 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:39.043464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd190877b4df34f3292d692199efe66d5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.044578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd190877b4df34f3292d692199efe66d5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.049625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:39.051111 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.057957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-b76d1789-8fc6-4194-8872-05e0badc6b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd190877b4df34f3292d692199efe66d5'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.073950 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1387/2767] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:38 2022] DELETE /identity/v3/users/d190877b4df34f3292d692199efe66d5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:39.078517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:39.078806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:39.079081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:39.079820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc72c70-deb6-4ee1-8260-afa4219900cb admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:39.097072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:39.098300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.109444 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:39.142472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:39.143351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:39.143627 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:39.143895 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] PATH_INFO: `/v3/projects/bdb61d2c0d744c9cac575aca41d9234b` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:39.145132 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb61d2c0d744c9cac575aca41d9234b)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:39.147078 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:39.163318 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1388/2768] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:39.173699 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:39.175236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.198372 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb61d2c0d744c9cac575aca41d9234b', 'request_id': 'req-4299759b-1425-4511-8f8e-739a1271caf1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.206753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb61d2c0d744c9cac575aca41d9234b', 'request_id': 'req-4299759b-1425-4511-8f8e-739a1271caf1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.212590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb61d2c0d744c9cac575aca41d9234b', 'request_id': 'req-4299759b-1425-4511-8f8e-739a1271caf1'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.224039 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] The token cache is being invalidate because project bdb61d2c0d744c9cac575aca41d9234b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:39.224325 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.225051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.226953 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1381/2769] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:39 2022] DELETE /identity/v3/projects/bdb61d2c0d744c9cac575aca41d9234b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:39.230768 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:39.233723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.235798 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f96b7-8a21-4209-97ba-bd6f595561d7 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:39.283423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:39.284084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:39.284366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:39.284662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:39.286453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.299878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:39.300106 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299759b-1425-4511-8f8e-739a1271caf1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:39.301418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:39.306152 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:39.307056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:39.307338 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:39.307668 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] PATH_INFO: `/v3/users/818841f0b8d341b98f82082c3f2529eb` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:39.309037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818841f0b8d341b98f82082c3f2529eb)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:39.311126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:39.311687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1389/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:39.371056 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818841f0b8d341b98f82082c3f2529eb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.371753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818841f0b8d341b98f82082c3f2529eb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.383123 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:39.385637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818841f0b8d341b98f82082c3f2529eb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.386860 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818841f0b8d341b98f82082c3f2529eb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.397333 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818841f0b8d341b98f82082c3f2529eb'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.410743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1382/2771] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:39 2022] DELETE /identity/v3/users/818841f0b8d341b98f82082c3f2529eb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:39.610079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:39.611554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.624955 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4315f-dbe9-476f-9928-5f3a5adcbe32 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:39.672971 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:39.698493 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:39.699502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:39.699890 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:39.700590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] PATH_INFO: `/v3/projects/9b35edebd9434f76a82d6281e0b4e631` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:39.702126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b35edebd9434f76a82d6281e0b4e631)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:39.704063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:39.755125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b35edebd9434f76a82d6281e0b4e631', 'request_id': 'req-f3a24301-8ab6-4191-8c77-4c642245b340'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.762035 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b35edebd9434f76a82d6281e0b4e631', 'request_id': 'req-f3a24301-8ab6-4191-8c77-4c642245b340'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.767979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b35edebd9434f76a82d6281e0b4e631', 'request_id': 'req-f3a24301-8ab6-4191-8c77-4c642245b340'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.782566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] The token cache is being invalidate because project 9b35edebd9434f76a82d6281e0b4e631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:39.782853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.783483 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.785187 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1390/2772] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:39 2022] DELETE /identity/v3/projects/9b35edebd9434f76a82d6281e0b4e631 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:39.789318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:39.790749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.804964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f044f-6094-4d0c-b55d-52ae3b7fd538 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:39.866887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:39.867793 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:39.868075 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:39.868349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] PATH_INFO: `/v3/users/fba49625bd05432f972631cea935b62a` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:39.869657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba49625bd05432f972631cea935b62a)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:39.871597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:39.926761 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba49625bd05432f972631cea935b62a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.927381 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba49625bd05432f972631cea935b62a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.940193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba49625bd05432f972631cea935b62a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.940876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba49625bd05432f972631cea935b62a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.949309 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-056a2715-f0f0-471a-a33d-a64970e535d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba49625bd05432f972631cea935b62a'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:39.964718 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1383/2773] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:39 2022] DELETE /identity/v3/users/fba49625bd05432f972631cea935b62a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:39.983532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:39.985061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:39.998085 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24301-8ab6-4191-8c77-4c642245b340 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.075715 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.077047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.077047 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.077282 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] PATH_INFO: `/v3/projects/3dac2bdc24d6463486025dd734eaadd0` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.078369 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dac2bdc24d6463486025dd734eaadd0)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.080317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.133596 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dac2bdc24d6463486025dd734eaadd0', 'request_id': 'req-ffeec78c-573b-4f91-97c8-8cb6ec7128de'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.140456 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dac2bdc24d6463486025dd734eaadd0', 'request_id': 'req-ffeec78c-573b-4f91-97c8-8cb6ec7128de'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.147353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dac2bdc24d6463486025dd734eaadd0', 'request_id': 'req-ffeec78c-573b-4f91-97c8-8cb6ec7128de'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.159057 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:40.163505 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] The token cache is being invalidate because project 3dac2bdc24d6463486025dd734eaadd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:40.164265 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.165566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.172263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1391/2774] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:39 2022] DELETE /identity/v3/projects/3dac2bdc24d6463486025dd734eaadd0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:40.177247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313fa00-59c9-476f-b197-a840eaeea077 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.177732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313fa00-59c9-476f-b197-a840eaeea077 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.177732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313fa00-59c9-476f-b197-a840eaeea077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.200073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-2313fa00-59c9-476f-b197-a840eaeea077 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:40.201364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-2313fa00-59c9-476f-b197-a840eaeea077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.253159 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:40.264170 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1384/2775] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:40.266363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.271723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.275557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.275935 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.276273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.285355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeec78c-573b-4f91-97c8-8cb6ec7128de admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.290363 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.291256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.291520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.291791 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] PATH_INFO: `/v3/projects/73e88ee5790c48b7a91c2fef947d2f9a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.293089 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e88ee5790c48b7a91c2fef947d2f9a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.295166 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.303635 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:40.304970 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.312965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1385/2776] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:40.317071 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.327326 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a722b5d-56bb-486d-9727-51cefa368e09 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.332104 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.332801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.333084 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.333359 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.334083 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:40.334387 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.335981 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.347879 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e88ee5790c48b7a91c2fef947d2f9a', 'request_id': 'req-d180c91d-e8f1-441b-9f25-5fba2604b7ad'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.348116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1386/2777] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:44:40.352590 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.355511 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e88ee5790c48b7a91c2fef947d2f9a', 'request_id': 'req-d180c91d-e8f1-441b-9f25-5fba2604b7ad'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.362306 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e88ee5790c48b7a91c2fef947d2f9a', 'request_id': 'req-d180c91d-e8f1-441b-9f25-5fba2604b7ad'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.362836 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb573-7cec-48d1-8d9b-994763d121b8 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.374735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] The token cache is being invalidate because project 73e88ee5790c48b7a91c2fef947d2f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:40.375026 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.375663 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-d180c91d-e8f1-441b-9f25-5fba2604b7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:40.377544 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1392/2778] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:40 2022] DELETE /identity/v3/projects/73e88ee5790c48b7a91c2fef947d2f9a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:40.382082 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e59a9-2534-4471-a417-c96e242c7f6a None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.382359 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e59a9-2534-4471-a417-c96e242c7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.382643 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e59a9-2534-4471-a417-c96e242c7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.401405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ef1e59a9-2534-4471-a417-c96e242c7f6a None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:40.402616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ef1e59a9-2534-4471-a417-c96e242c7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.421321 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.422232 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.422361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.422595 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.423330 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.425271 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.452665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1387/2779] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:40.456126 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.457818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.465710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1393/2780] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:40.473912 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.474200 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.474478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.493614 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:40.494878 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.521938 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee69efe-d697-4b38-8626-ecda88e1da29 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.527134 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.527971 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.528357 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.528655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.529348 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.531564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.557217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1388/2781] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:40.558257 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.559350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1394/2782] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:40.564314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.568161 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818753fd-61a1-4c49-a230-fbe572a250e7 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.573250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.573706 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745421d-f462-4c91-a2e8-68ae132253de None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.574186 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.574539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.574909 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.575715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.577478 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.578590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.579245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.579527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.579806 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.580539 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:40.580883 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.582478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.588820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1389/2783] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:40.593573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.594963 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1395/2784] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:40.598953 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.603655 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16162b65-a36a-4f32-9e91-88c8a78f187a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.608819 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909424c6-a10e-4c92-9637-30abc875e861 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.609138 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.609801 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.610080 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.610353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.611073 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.612702 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.613578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.614224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.614506 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.614836 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.615527 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.617475 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.623349 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1390/2785] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:40.626228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:40.627505 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.639196 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604debe8-af06-4bcc-9648-729f7f1aba3c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.644308 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf44573-5471-45fa-8462-3c6884242961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.645235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf44573-5471-45fa-8462-3c6884242961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.645512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf44573-5471-45fa-8462-3c6884242961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.645666 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1396/2786] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:40.645840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf44573-5471-45fa-8462-3c6884242961 admin admin] PATH_INFO: `/v3/projects/b9ebc78f23044b98a13e5e3ebc5dd614/users/6b952a9d6bad40c4b2829256bed0e017/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.651039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.662096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf44573-5471-45fa-8462-3c6884242961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ebc78f23044b98a13e5e3ebc5dd614, user_id=6b952a9d6bad40c4b2829256bed0e017, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.664888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf44573-5471-45fa-8462-3c6884242961 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.678908 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1391/2787] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:40 2022] PUT /identity/v3/projects/b9ebc78f23044b98a13e5e3ebc5dd614/users/6b952a9d6bad40c4b2829256bed0e017/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:40.683925 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.684276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.684651 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.696177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:40.702523 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:40.704706 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.717944 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a6eb8-c1c6-4f10-81a7-e165b09a9d2c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.765928 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1392/2788] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:40.776019 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.777423 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:40.777984 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.778786 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.779063 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.779337 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.780029 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.782079 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.803560 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1397/2789] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:40.807386 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.817300 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022ffb-af6b-4975-b691-90d503f8a877 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.822007 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.822672 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.822954 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.823230 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.823960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.825690 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.835942 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1398/2790] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:40.839718 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.841264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2b93-3546-4d04-be48-7cb447c4c058 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.844888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.845541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.845820 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.846090 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.849314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090c512-cad1-47eb-ace4-74e0e262bfae admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.854284 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10438f-5db5-4e6d-9b11-1d3947039c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.854943 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10438f-5db5-4e6d-9b11-1d3947039c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.855245 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10438f-5db5-4e6d-9b11-1d3947039c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.855375 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.855525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10438f-5db5-4e6d-9b11-1d3947039c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.856278 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10438f-5db5-4e6d-9b11-1d3947039c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.857052 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.858028 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10438f-5db5-4e6d-9b11-1d3947039c0c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.866887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1393/2791] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:44:40.872168 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1399/2792] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:40.873371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:40.883324 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0710b5-5af1-4eb4-bcf0-f2596be81313 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:40.887978 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:40.889098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.889384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.889686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] PATH_INFO: `/v3/projects/317f1ae156db41df9fd0feaeb0315cbc/users/a8c2a2da930d4190b157380f7de6d3c6/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:40.905546 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317f1ae156db41df9fd0feaeb0315cbc, user_id=a8c2a2da930d4190b157380f7de6d3c6, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:40.908212 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:40.921715 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1394/2793] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:40 2022] PUT /identity/v3/projects/317f1ae156db41df9fd0feaeb0315cbc/users/a8c2a2da930d4190b157380f7de6d3c6/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:40.987413 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:40.987705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:40.987986 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.016366 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] MFA Rules not processed for user `a8c2a2da930d4190b157380f7de6d3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:41.017658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:41.081428 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:41.096070 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1400/2794] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:41.100512 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.186529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78838529-fff1-4cc6-b6f9-64c07bce8e57 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.192370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.193358 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.193858 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.194204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.195095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.197136 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.232681 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1395/2795] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:41 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:41.234658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.246147 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd5c83-ba9d-411c-836f-86b0fa9249f1 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.312780 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.313464 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.313743 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.314013 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.314722 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.316695 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.341862 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1401/2796] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:41 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:41.343487 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.354665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85b68a-1c9c-4c04-ab77-332c33dd40ab admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.359686 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.360384 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.360699 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.360982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.361762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.363411 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.370277 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:41.378329 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1396/2797] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:41.382261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.394870 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc318-ec7b-4d30-bd5c-1994efbbe0ff admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.400135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.401312 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.401424 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.401603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] PATH_INFO: `/v3/projects/268eea09e57d44e6bbe61c0316c41a56/users/63051bdb7292479aba31c3bc2109d0f7/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.419528 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268eea09e57d44e6bbe61c0316c41a56, user_id=63051bdb7292479aba31c3bc2109d0f7, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.423169 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.444905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1402/2798] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:41 2022] PUT /identity/v3/projects/268eea09e57d44e6bbe61c0316c41a56/users/63051bdb7292479aba31c3bc2109d0f7/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:41.446178 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.458784 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f086f-22a1-4267-9ba6-888f513cdefc admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.523989 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a503-ae5f-43ef-92ad-383b31a61469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.524743 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a503-ae5f-43ef-92ad-383b31a61469 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.525027 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a503-ae5f-43ef-92ad-383b31a61469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.525273 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a503-ae5f-43ef-92ad-383b31a61469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.526014 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a503-ae5f-43ef-92ad-383b31a61469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.527634 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a503-ae5f-43ef-92ad-383b31a61469 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.539914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1397/2799] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:41.544256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.558073 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b08-6bf5-475c-bfe9-8bb6ecda6b01 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.563327 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.564261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.564545 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.564850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] PATH_INFO: `/v3/projects/268eea09e57d44e6bbe61c0316c41a56/users/63051bdb7292479aba31c3bc2109d0f7/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.567022 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268eea09e57d44e6bbe61c0316c41a56, user_id=63051bdb7292479aba31c3bc2109d0f7, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.569753 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.581728 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1403/2800] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:41 2022] PUT /identity/v3/projects/268eea09e57d44e6bbe61c0316c41a56/users/63051bdb7292479aba31c3bc2109d0f7/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:41.593366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e18a0-b735-4d70-84de-7aa5f30b69cc None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.593601 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e18a0-b735-4d70-84de-7aa5f30b69cc None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.593914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e18a0-b735-4d70-84de-7aa5f30b69cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.623902 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a66e18a0-b735-4d70-84de-7aa5f30b69cc None None] MFA Rules not processed for user `63051bdb7292479aba31c3bc2109d0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:41.625786 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a66e18a0-b735-4d70-84de-7aa5f30b69cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:41.707995 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1398/2801] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:41.722106 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.724347 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:41.809082 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.809578 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.810053 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.816478 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7330-e05a-4ca0-899c-353cbfd4bb41 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.820357 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.821104 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.821387 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.821904 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.832148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.833662 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.841709 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] MFA Rules not processed for user `6b952a9d6bad40c4b2829256bed0e017`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:41.843016 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:41.845670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1404/2802] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:41.897016 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.900529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:41.912900 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51be0-70ef-436e-94f9-6d7c7e26bcad service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.917368 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef074c-0b9e-43f7-854b-b04d91e24a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.918488 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef074c-0b9e-43f7-854b-b04d91e24a4b service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.918832 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef074c-0b9e-43f7-854b-b04d91e24a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.919122 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef074c-0b9e-43f7-854b-b04d91e24a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.943472 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1399/2803] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:41.953495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:41.967831 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bdb84-d9f2-4e96-a3b3-c92bd683999c None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:41.971820 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:41.976968 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:41.978285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:41.978670 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:41.979061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:41.987629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef074c-0b9e-43f7-854b-b04d91e24a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.989207 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef074c-0b9e-43f7-854b-b04d91e24a4b service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.996103 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:41.998197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:41.999962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1405/2804] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:42.002901 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=100 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:42.010219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1400/2805] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:43.420956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a36ab-6d8e-4b76-b2ff-f0560eae9d1f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:43.420956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a36ab-6d8e-4b76-b2ff-f0560eae9d1f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:43.420956 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a36ab-6d8e-4b76-b2ff-f0560eae9d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:43.434514 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-357a36ab-6d8e-4b76-b2ff-f0560eae9d1f None None] MFA Rules not processed for user `6b952a9d6bad40c4b2829256bed0e017`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:43.435754 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-357a36ab-6d8e-4b76-b2ff-f0560eae9d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:43.443568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1406/2806] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:45.456406 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:45.519558 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acafc95-57c4-4b51-b4ed-fa80aa27528e service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:45.524929 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:45.526328 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:45.526607 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:45.526879 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] PATH_INFO: `/v3/users/a8c2a2da930d4190b157380f7de6d3c6` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:45.528204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c2a2da930d4190b157380f7de6d3c6)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:45.530246 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:45.582524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c2a2da930d4190b157380f7de6d3c6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:45.583177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c2a2da930d4190b157380f7de6d3c6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:45.595712 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c2a2da930d4190b157380f7de6d3c6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:45.596368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c2a2da930d4190b157380f7de6d3c6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:45.605583 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c2a2da930d4190b157380f7de6d3c6'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:45.619664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1401/2807] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:45 2022] DELETE /identity/v3/users/a8c2a2da930d4190b157380f7de6d3c6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:45.624039 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:45.624735 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:45.625038 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:45.647271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:45.648857 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:45.712566 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1407/2808] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:45.722115 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:45.723527 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:45.782876 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31f688-5cbe-4748-98b0-984a390384ff admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:45.787254 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:45.787905 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:45.788182 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:45.788464 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:45.798173 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:45.799614 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:45.809753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1402/2809] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:44:46.297352 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:46.298850 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:46.311370 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aef5b-dfb3-4c9b-be60-b2d87e23c5c8 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:46.322837 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:46.323964 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:46.324421 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:46.325310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] PATH_INFO: `/v3/projects/317f1ae156db41df9fd0feaeb0315cbc` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:46.327042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=317f1ae156db41df9fd0feaeb0315cbc)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:46.336280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:46.412532 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317f1ae156db41df9fd0feaeb0315cbc', 'request_id': 'req-1d6b8d4f-d21f-4a47-827f-b740042ff42c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.418975 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317f1ae156db41df9fd0feaeb0315cbc', 'request_id': 'req-1d6b8d4f-d21f-4a47-827f-b740042ff42c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.424930 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317f1ae156db41df9fd0feaeb0315cbc', 'request_id': 'req-1d6b8d4f-d21f-4a47-827f-b740042ff42c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.441378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] The token cache is being invalidate because project 317f1ae156db41df9fd0feaeb0315cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:46.441828 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.442573 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.444193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1408/2810] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:46 2022] DELETE /identity/v3/projects/317f1ae156db41df9fd0feaeb0315cbc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:46.448363 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:46.459087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2788d-d962-44f4-bf13-908f822ef717 service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:46.477750 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:46.520999 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:46.521888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:46.522164 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:46.522435 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] PATH_INFO: `/v3/users/63051bdb7292479aba31c3bc2109d0f7` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:46.524029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63051bdb7292479aba31c3bc2109d0f7)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:46.525988 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:46.578048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63051bdb7292479aba31c3bc2109d0f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.578956 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63051bdb7292479aba31c3bc2109d0f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.596687 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63051bdb7292479aba31c3bc2109d0f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.597680 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63051bdb7292479aba31c3bc2109d0f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.607638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-3ed86e87-5b8d-4ccc-89f9-a4b00c59c258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63051bdb7292479aba31c3bc2109d0f7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:46.622045 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1403/2811] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:46 2022] DELETE /identity/v3/users/63051bdb7292479aba31c3bc2109d0f7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:46.910941 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:46.912512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:46.934279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b8d4f-d21f-4a47-827f-b740042ff42c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:46.994901 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:46.995705 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:46.995983 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:46.996280 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] PATH_INFO: `/v3/projects/268eea09e57d44e6bbe61c0316c41a56` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:46.997520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268eea09e57d44e6bbe61c0316c41a56)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:46.999649 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.006271 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:47.050095 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268eea09e57d44e6bbe61c0316c41a56', 'request_id': 'req-0c423bba-6aa2-4d1b-a804-84211e67be56'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:47.056266 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268eea09e57d44e6bbe61c0316c41a56', 'request_id': 'req-0c423bba-6aa2-4d1b-a804-84211e67be56'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:47.061808 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268eea09e57d44e6bbe61c0316c41a56', 'request_id': 'req-0c423bba-6aa2-4d1b-a804-84211e67be56'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:47.076710 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] The token cache is being invalidate because project 268eea09e57d44e6bbe61c0316c41a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:47.077032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:47.077628 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-0c423bba-6aa2-4d1b-a804-84211e67be56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:47.079258 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1409/2812] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:46 2022] DELETE /identity/v3/projects/268eea09e57d44e6bbe61c0316c41a56 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:47.084176 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.084465 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.084798 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.102429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:47.103648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:47.165580 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1404/2813] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:47.173709 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.174006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.174275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.193905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:47.195154 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:47.204150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1410/2814] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:47.208753 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.219525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b404655-40f2-427d-af37-cfd3ee1082e1 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.224797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.225490 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.225749 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.226017 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.226735 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:47.227037 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.228665 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.241105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1405/2815] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:47.245501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.255647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1af61-c7d6-4ccb-95ae-0d18b1ee3f47 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.260917 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.261569 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.261852 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.262125 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.262824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.264864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.294336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1411/2816] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:47.298799 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.309312 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4c0ec-f78f-46b6-82ea-970851656ffe admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.336656 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:47.372031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.372731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.373012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.373280 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.374157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.376065 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.396317 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1406/2817] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:47.401050 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.412425 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef927-975c-4675-b7a1-39934e6f64bf admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.418414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.419066 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.419391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.419993 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.420406 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.422121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.433646 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1412/2818] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:47.438285 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.448564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb202bf-abe0-42bc-982a-dc02d5f03f6d admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.454217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1867897-48c8-405a-9b6e-aeb9e07c8f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.454897 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1867897-48c8-405a-9b6e-aeb9e07c8f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.455181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1867897-48c8-405a-9b6e-aeb9e07c8f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.455455 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1867897-48c8-405a-9b6e-aeb9e07c8f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.456197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1867897-48c8-405a-9b6e-aeb9e07c8f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.457828 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1867897-48c8-405a-9b6e-aeb9e07c8f0a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.469260 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1407/2819] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:47.473977 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.484891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c0f5d-b7e1-4585-941e-b46966b19451 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.490150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.491851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.492134 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.492411 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] PATH_INFO: `/v3/projects/6abc0c207b1844d59475200f7685c8c3/users/89efc0097d9a45299bcc8db89ecc903a/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.509578 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6abc0c207b1844d59475200f7685c8c3, user_id=89efc0097d9a45299bcc8db89ecc903a, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.511848 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.526554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1413/2820] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:47 2022] PUT /identity/v3/projects/6abc0c207b1844d59475200f7685c8c3/users/89efc0097d9a45299bcc8db89ecc903a/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:47.538077 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.538368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.538649 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.567291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] MFA Rules not processed for user `89efc0097d9a45299bcc8db89ecc903a`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:47.568524 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:47.633543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1408/2821] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:47.638648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.706471 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dacc9-054d-441c-93a6-dc9977ed1b5b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.712302 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.713021 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.713305 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.713574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.714275 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.716211 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.746236 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1414/2822] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:47.750503 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.761758 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ebbc2-ebaf-471a-9220-38b47cb7f581 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.820207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.820925 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.821205 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.821477 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.822157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.824088 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.848782 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1409/2823] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:47 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:47.850342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.861921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1cd38-dde9-4efa-b1e3-374d497141df admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.867540 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.868217 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.868495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.868792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.869520 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.871099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.882172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1415/2824] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:47.886368 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.897410 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b58a2-4586-432b-8731-7f029f378267 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:47.902818 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:47.903723 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:47.903998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:47.904264 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] PATH_INFO: `/v3/projects/3e5a9bca693d43e0bfb762fb1379cd3c/users/6390a40f5a504512b2b50130f02df8ed/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:47.919814 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a9bca693d43e0bfb762fb1379cd3c, user_id=6390a40f5a504512b2b50130f02df8ed, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:47.922921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:47.932661 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:47.942929 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:47.943450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1410/2825] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:47 2022] PUT /identity/v3/projects/3e5a9bca693d43e0bfb762fb1379cd3c/users/6390a40f5a504512b2b50130f02df8ed/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:47.950009 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:47.954224 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178f39-9089-4e02-aded-112852077e57 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:48.015594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018962-2198-488a-8bf8-c265f685737d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:48.016191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018962-2198-488a-8bf8-c265f685737d admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.016473 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018962-2198-488a-8bf8-c265f685737d admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.016792 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018962-2198-488a-8bf8-c265f685737d admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.017525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63018962-2198-488a-8bf8-c265f685737d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:48.019139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63018962-2198-488a-8bf8-c265f685737d admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:48.029612 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1416/2826] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:44:48.033789 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:48.044542 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83df705-3e21-4014-9c8a-30f7ac0ca19c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:48.049557 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:48.050492 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.050774 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.051048 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] PATH_INFO: `/v3/projects/3e5a9bca693d43e0bfb762fb1379cd3c/users/6390a40f5a504512b2b50130f02df8ed/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.053099 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a9bca693d43e0bfb762fb1379cd3c, user_id=6390a40f5a504512b2b50130f02df8ed, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:48.055751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:48.067215 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1411/2827] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:48 2022] PUT /identity/v3/projects/3e5a9bca693d43e0bfb762fb1379cd3c/users/6390a40f5a504512b2b50130f02df8ed/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:48.077606 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.077922 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.078170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.105770 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] MFA Rules not processed for user `6390a40f5a504512b2b50130f02df8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:48.107006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:48.180951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1417/2828] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:48.192916 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:48.194732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:48.252940 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05eb4-4371-49d9-9c8c-4a2b1613ac3f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:48.256566 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0de33-09ac-4b86-a62e-b2ac5da1721f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:48.257250 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0de33-09ac-4b86-a62e-b2ac5da1721f service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.257531 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0de33-09ac-4b86-a62e-b2ac5da1721f service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.257804 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0de33-09ac-4b86-a62e-b2ac5da1721f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.267253 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0de33-09ac-4b86-a62e-b2ac5da1721f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:48.268657 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0de33-09ac-4b86-a62e-b2ac5da1721f service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:48.278378 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1412/2829] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:48.342868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:48.409684 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16050-e1db-4ca5-8457-a8ef027a086f None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:48.414518 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:48.415405 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.415687 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.415960 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] PATH_INFO: `/v3/users/89efc0097d9a45299bcc8db89ecc903a` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.417210 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89efc0097d9a45299bcc8db89ecc903a)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:48.419148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:48.450027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:48.471664 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89efc0097d9a45299bcc8db89ecc903a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:48.472307 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89efc0097d9a45299bcc8db89ecc903a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:48.485060 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89efc0097d9a45299bcc8db89ecc903a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:48.485823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89efc0097d9a45299bcc8db89ecc903a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:48.494496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89efc0097d9a45299bcc8db89ecc903a'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:48.508126 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1418/2830] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:48 2022] DELETE /identity/v3/users/89efc0097d9a45299bcc8db89ecc903a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:48.513116 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.513638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.513638 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.532462 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:48.533716 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:48.595306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1413/2831] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:48.605182 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:48.606772 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:48.666671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7ba9a-06bb-41fc-b202-61532fe9cb18 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:48.671385 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:48.672061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:48.672345 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:48.672659 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:48.683043 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:48.684531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:48.694839 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1419/2832] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:49.017888 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:49.019532 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:49.033098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe1439-e581-44ae-8ba2-8d8df14b49e9 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:49.038642 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:49.039529 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.039807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.040098 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] PATH_INFO: `/v3/projects/6abc0c207b1844d59475200f7685c8c3` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.041440 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6abc0c207b1844d59475200f7685c8c3)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:49.043351 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:49.094029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6abc0c207b1844d59475200f7685c8c3', 'request_id': 'req-c2da95e0-8d06-4dbe-b836-8af3ad39a577'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.121204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6abc0c207b1844d59475200f7685c8c3', 'request_id': 'req-c2da95e0-8d06-4dbe-b836-8af3ad39a577'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.129024 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6abc0c207b1844d59475200f7685c8c3', 'request_id': 'req-c2da95e0-8d06-4dbe-b836-8af3ad39a577'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.142775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] The token cache is being invalidate because project 6abc0c207b1844d59475200f7685c8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:49.143061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.145827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.148405 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1414/2833] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:49 2022] DELETE /identity/v3/projects/6abc0c207b1844d59475200f7685c8c3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:49.153292 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:49.168939 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d606f47-0619-479c-b1fd-328634449743 service neutron] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:49.227974 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:49.228888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.229167 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.229437 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] PATH_INFO: `/v3/users/6390a40f5a504512b2b50130f02df8ed` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.230648 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6390a40f5a504512b2b50130f02df8ed)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:49.232559 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:49.282225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6390a40f5a504512b2b50130f02df8ed'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.282937 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6390a40f5a504512b2b50130f02df8ed'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.285317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:49.298177 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6390a40f5a504512b2b50130f02df8ed'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.298824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6390a40f5a504512b2b50130f02df8ed'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.306109 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-dd778c4c-9a35-469e-9727-e3792b8c6934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6390a40f5a504512b2b50130f02df8ed'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.318315 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1420/2834] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:49 2022] DELETE /identity/v3/users/6390a40f5a504512b2b50130f02df8ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:49.619684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:49.621287 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:49.638033 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da95e0-8d06-4dbe-b836-8af3ad39a577 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:49.649353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:49.701192 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:49.702356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.702637 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.702907 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] PATH_INFO: `/v3/projects/3e5a9bca693d43e0bfb762fb1379cd3c` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.704095 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5a9bca693d43e0bfb762fb1379cd3c)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:49.706029 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:49.755528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5a9bca693d43e0bfb762fb1379cd3c', 'request_id': 'req-f79bad6c-08a9-41c7-851a-b741baada639'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.764771 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5a9bca693d43e0bfb762fb1379cd3c', 'request_id': 'req-f79bad6c-08a9-41c7-851a-b741baada639'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.770816 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5a9bca693d43e0bfb762fb1379cd3c', 'request_id': 'req-f79bad6c-08a9-41c7-851a-b741baada639'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.782449 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] The token cache is being invalidate because project 3e5a9bca693d43e0bfb762fb1379cd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:49.782732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.783388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f79bad6c-08a9-41c7-851a-b741baada639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:49.784973 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1415/2835] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:49 2022] DELETE /identity/v3/projects/3e5a9bca693d43e0bfb762fb1379cd3c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:49.791336 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.791635 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.791920 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.810555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:49.811934 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:49.871868 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1421/2836] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:49.881177 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.881468 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.881748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.901461 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:49.902684 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:49.910873 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1416/2837] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:49.915232 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:49.925603 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e3ade-c327-4c70-9093-54184a72e162 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:49.930174 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:49.930844 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.931121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.931394 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] PATH_INFO: `/v3/domains` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.932127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80097) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:49.932422 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:49.934061 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:49.946087 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1422/2838] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:49.950553 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:49.960207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2380-15a0-420f-a915-5ca6a69ce9cf None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:49.964998 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:49.965648 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:49.965927 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:49.966198 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:49.966882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:49.968899 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:49.995235 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1417/2839] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:49 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:50.000078 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.015335 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4b3fe-0fd2-4890-bfea-ebec9e53e53b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.075293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.075988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.076272 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.076547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.077355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.079361 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.100647 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1423/2840] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:49 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:50.104568 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.116399 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485578b-90f0-42cc-8322-5f6d2bf64954 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.121563 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.122269 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.122559 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.122833 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.123592 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.125288 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.137344 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1418/2841] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:50.141496 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.151995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f91cd2-9050-4e23-8881-73fded7460dc admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.157172 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d13744-5dec-40c8-b10c-2e4672273a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.157853 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d13744-5dec-40c8-b10c-2e4672273a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.158135 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d13744-5dec-40c8-b10c-2e4672273a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.158423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d13744-5dec-40c8-b10c-2e4672273a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.159184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d13744-5dec-40c8-b10c-2e4672273a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.160824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d13744-5dec-40c8-b10c-2e4672273a7b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.174429 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1424/2842] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:50.175626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.186174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51d0a-5d02-4f2a-94c3-c9e9aa97c0ce admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.191240 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.192142 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.192425 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.192797 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] PATH_INFO: `/v3/projects/d71b297847b04e729cc40ab42c950919/users/00cf37ffc3594e848dacd714d64ba971/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.208834 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71b297847b04e729cc40ab42c950919, user_id=00cf37ffc3594e848dacd714d64ba971, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.211573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.226157 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1419/2843] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:50 2022] PUT /identity/v3/projects/d71b297847b04e729cc40ab42c950919/users/00cf37ffc3594e848dacd714d64ba971/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:50.236679 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.237051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.237334 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.265923 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] MFA Rules not processed for user `00cf37ffc3594e848dacd714d64ba971`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:50.267185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:50.290634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:50.346504 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1425/2844] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:50.351181 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.397225 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:50.419154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bd5a-d66b-4d46-8d2c-9cfa4f74821e admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.424154 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.424862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.425141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.425413 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] PATH_INFO: `/v3/projects` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.426076 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.427982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.455012 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1420/2845] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:50 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:50.461767 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.471894 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee0b13-c680-4014-93ca-e0ad6181b51b None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.528150 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.528864 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.529139 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.529409 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] PATH_INFO: `/v3/users` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.530714 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.532721 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.553252 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1426/2846] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:50 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:50.557436 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.568942 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab1bcf-6b0f-4e0e-bb04-50056346e59f admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.573937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.574539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.574861 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.575133 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.575949 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.577626 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.590481 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1421/2847] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:50.595027 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.605613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcc930-3982-4c36-b9e6-e0a57902de9c admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.611198 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.612113 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.612393 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.612682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] PATH_INFO: `/v3/projects/e7ad9ab09b6245678a8a4ceba05a6722/users/d1186cfd8c144b2daa56ddfb1497711d/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.628381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ad9ab09b6245678a8a4ceba05a6722, user_id=d1186cfd8c144b2daa56ddfb1497711d, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.631083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.646668 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1427/2848] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:50 2022] PUT /identity/v3/projects/e7ad9ab09b6245678a8a4ceba05a6722/users/d1186cfd8c144b2daa56ddfb1497711d/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:50.651067 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.661295 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a77500-718b-40f1-82f4-db93c5962b13 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.719937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c8592-1f28-473b-842a-23924c543243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.720582 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c8592-1f28-473b-842a-23924c543243 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.720887 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c8592-1f28-473b-842a-23924c543243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.721174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c8592-1f28-473b-842a-23924c543243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.721900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c8592-1f28-473b-842a-23924c543243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.723544 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c8592-1f28-473b-842a-23924c543243 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.733420 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1422/2849] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:50.737825 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.748378 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075f52-4860-40fc-8b72-3db2706de47a admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.753353 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.754255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.754533 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.754809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] PATH_INFO: `/v3/projects/e7ad9ab09b6245678a8a4ceba05a6722/users/d1186cfd8c144b2daa56ddfb1497711d/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.756940 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ad9ab09b6245678a8a4ceba05a6722, user_id=d1186cfd8c144b2daa56ddfb1497711d, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.760584 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:50.772314 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1428/2850] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:50 2022] PUT /identity/v3/projects/e7ad9ab09b6245678a8a4ceba05a6722/users/d1186cfd8c144b2daa56ddfb1497711d/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:50.783261 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.783619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.783900 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.816839 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] MFA Rules not processed for user `d1186cfd8c144b2daa56ddfb1497711d`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:50.818619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:50.905429 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1423/2851] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:50.916682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:50.918842 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:50.981114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685743e-a5ef-4447-958b-ef09bd595d2b admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:50.985703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:50.986444 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:50.986719 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:50.987074 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:50.996463 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:50.997962 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:51.008101 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1429/2852] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:51.118653 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:51.129209 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7675d6-9803-47df-8b9d-d3756323f90f None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:51.133093 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:51.134912 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:51.135268 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:51.135616 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:51.169217 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:51.195829 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:51.197320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:51.206213 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1424/2853] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:52.257151 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:52.259196 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:52.286657 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:52.364184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa196a-e775-448c-ac69-c0a04e2fec87 service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:52.370296 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ca71f32f61e41288cac452e58b4a1dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:52.370951 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:52.371231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:52.371503 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:52.373557 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:52.386826 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:52.388418 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873fac1-2eb6-44e5-b4ec-16439c8f4e0c service nova] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:52.399592 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1430/2854] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:53.804573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:53.875694 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c012fbd-5578-4881-90b6-83a3c30d0743 service cinder] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:53.881149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:53.882102 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:53.882386 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:53.882658 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] PATH_INFO: `/v3/users/00cf37ffc3594e848dacd714d64ba971` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:53.884119 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cf37ffc3594e848dacd714d64ba971)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:53.886130 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:53.941117 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cf37ffc3594e848dacd714d64ba971'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:53.941779 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cf37ffc3594e848dacd714d64ba971'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:53.956042 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cf37ffc3594e848dacd714d64ba971'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:53.956964 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cf37ffc3594e848dacd714d64ba971'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:53.969135 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cf37ffc3594e848dacd714d64ba971'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:53.982776 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1425/2855] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:53 2022] DELETE /identity/v3/users/00cf37ffc3594e848dacd714d64ba971 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:53.988568 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:53.989121 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:53.989233 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:54.012547 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:54.013863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:54.077432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1431/2856] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:54.087756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:54.089554 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:54.152107 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf214-c217-4e90-a168-aff9bfca6909 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:54.156150 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:54.156840 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:54.157124 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:54.157404 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:54.164729 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:54.167904 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:54.174673 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:54.182488 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1426/2857] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:54.462048 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:54.464033 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:54.477670 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359073f-08a2-4cbf-9c1a-2c01450296be None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:54.482995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:54.484231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:54.485076 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:54.485407 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] PATH_INFO: `/v3/projects/d71b297847b04e729cc40ab42c950919` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:54.486841 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71b297847b04e729cc40ab42c950919)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:54.488804 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:54.548164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71b297847b04e729cc40ab42c950919', 'request_id': 'req-4fd79466-86d8-431f-b6f1-42dbb7c72633'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.555494 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71b297847b04e729cc40ab42c950919', 'request_id': 'req-4fd79466-86d8-431f-b6f1-42dbb7c72633'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.563231 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71b297847b04e729cc40ab42c950919', 'request_id': 'req-4fd79466-86d8-431f-b6f1-42dbb7c72633'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.584586 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] The token cache is being invalidate because project d71b297847b04e729cc40ab42c950919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:54.584840 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.585484 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.587432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1432/2858] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:54 2022] DELETE /identity/v3/projects/d71b297847b04e729cc40ab42c950919 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:54.591496 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:54.604247 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a513-e9a6-4d23-b4c5-9191cf8cd2bf service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:54.661599 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:54.662493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:54.662775 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:54.663047 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] PATH_INFO: `/v3/users/d1186cfd8c144b2daa56ddfb1497711d` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:54.664245 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1186cfd8c144b2daa56ddfb1497711d)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:54.666191 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:54.715141 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1186cfd8c144b2daa56ddfb1497711d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.715790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1186cfd8c144b2daa56ddfb1497711d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.729647 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1186cfd8c144b2daa56ddfb1497711d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.730306 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1186cfd8c144b2daa56ddfb1497711d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.739370 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-f8dadb39-bcaf-495b-a9d2-9700a65aac21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1186cfd8c144b2daa56ddfb1497711d'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:54.753004 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1427/2859] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:44:54 2022] DELETE /identity/v3/users/d1186cfd8c144b2daa56ddfb1497711d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:55.035482 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.037081 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:55.050979 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79466-86d8-431f-b6f1-42dbb7c72633 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.078609 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:55.116423 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.117480 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.117851 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.118191 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] PATH_INFO: `/v3/projects/e7ad9ab09b6245678a8a4ceba05a6722` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.119683 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ad9ab09b6245678a8a4ceba05a6722)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.121824 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.176580 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ad9ab09b6245678a8a4ceba05a6722', 'request_id': 'req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:55.183775 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ad9ab09b6245678a8a4ceba05a6722', 'request_id': 'req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:55.189633 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ad9ab09b6245678a8a4ceba05a6722', 'request_id': 'req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:55.207574 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] The token cache is being invalidate because project e7ad9ab09b6245678a8a4ceba05a6722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:44:55.207899 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:55.208616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-e81fd07a-ea8a-4f37-b356-bfed7b54cb8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:44:55.210397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1433/2860] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:44:55 2022] DELETE /identity/v3/projects/e7ad9ab09b6245678a8a4ceba05a6722 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:55.218051 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.218926 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.219097 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.242679 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:55.243913 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:55.315874 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1428/2861] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:55.324957 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.325242 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.325517 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.345522 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:55.346774 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:55.354874 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1434/2862] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:44:55.359389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.369453 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90576609-51ff-4c50-96d9-21912281f7dd None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.374539 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.375229 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.375528 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.375803 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] PATH_INFO: `/v3/domains` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.376541 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80096) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:44:55.376892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.378537 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.391031 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1429/2863] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jun 30 12:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:44:55.395590 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.406051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baba8b4-526a-4335-878b-c54becc89e40 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.411261 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.412034 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.412389 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.412698 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.413432 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.415346 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.443281 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1435/2864] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:55.447851 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.453738 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:55.459361 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055d809-3ff4-4869-ac8b-c5245d4e8904 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.519431 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.520109 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.520389 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.520697 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.521397 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.523353 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.544990 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1430/2865] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:55.549310 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.559995 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d112307-31ca-41f6-b187-f111412be6e3 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.565225 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.565886 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.566185 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.566460 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.567184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.568872 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.579634 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1436/2866] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:55.583619 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.594415 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667be5-d9c8-4336-a389-5e1ef78ce5ca admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.599872 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e039fd-ada0-492c-a071-05f7be6f91d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.600547 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e039fd-ada0-492c-a071-05f7be6f91d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.600865 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e039fd-ada0-492c-a071-05f7be6f91d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.601146 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e039fd-ada0-492c-a071-05f7be6f91d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.601882 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e039fd-ada0-492c-a071-05f7be6f91d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.603508 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e039fd-ada0-492c-a071-05f7be6f91d2 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.614061 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1431/2867] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:55.618148 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.630223 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90afe54-85b0-4e4e-b6ac-d7ffff77d9c1 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.635180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.636072 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.636350 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.636621 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] PATH_INFO: `/v3/projects/728a8430417f4faf8ce27400ab7a4c89/users/0520c4d9bbc24afebe791f7e152e9405/roles/52504f5e779147cd8e9aa136bcd0ceae` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.652891 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=728a8430417f4faf8ce27400ab7a4c89, user_id=0520c4d9bbc24afebe791f7e152e9405, role_id=52504f5e779147cd8e9aa136bcd0ceae)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.655816 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.671537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1437/2868] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:55 2022] PUT /identity/v3/projects/728a8430417f4faf8ce27400ab7a4c89/users/0520c4d9bbc24afebe791f7e152e9405/roles/52504f5e779147cd8e9aa136bcd0ceae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:44:55.687564 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.687827 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.688105 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.714473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] MFA Rules not processed for user `0520c4d9bbc24afebe791f7e152e9405`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:55.715688 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:55.783314 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1432/2869] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:55.788194 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.848006 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:55.860523 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d253e-9a59-486c-8a33-a0f29285fdfd admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.866388 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.867000 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.867279 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.867548 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.868238 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.870209 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.896702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1438/2870] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:44:55.901376 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:55.912746 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa6f3-c645-480b-bb53-836278d3b207 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:55.974543 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:55.975149 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:55.975416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:55.975696 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:55.976371 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:55.978339 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:55.995690 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:56.001982 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1433/2871] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jun 30 12:44:55 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:44:56.006051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:56.016931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae03d9a-e061-4542-9b2c-2024ebd0bad1 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:56.022702 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:56.023373 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.024014 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.024293 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.025051 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:56.026682 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:56.037381 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1439/2872] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:56.041572 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:56.052615 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d34e1-70e1-4dda-97f3-d03c6b896d66 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:56.058252 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:56.059147 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.059430 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.059732 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] PATH_INFO: `/v3/projects/04570604c09c44a0be08f3729fa06e9e/users/afe61a071ba942c090e5b866743312d7/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.075466 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04570604c09c44a0be08f3729fa06e9e, user_id=afe61a071ba942c090e5b866743312d7, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:56.078230 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:56.094057 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1434/2873] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:56 2022] PUT /identity/v3/projects/04570604c09c44a0be08f3729fa06e9e/users/afe61a071ba942c090e5b866743312d7/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:56.098170 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:56.108555 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71624f93-0ff3-469e-bdfd-ea0d35f4a7ae admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:56.173502 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805f5bf-d236-4008-a509-09b4ee47dec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:56.174105 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805f5bf-d236-4008-a509-09b4ee47dec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.174384 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805f5bf-d236-4008-a509-09b4ee47dec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.174658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805f5bf-d236-4008-a509-09b4ee47dec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.175501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805f5bf-d236-4008-a509-09b4ee47dec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:56.177263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805f5bf-d236-4008-a509-09b4ee47dec8 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:56.188317 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1440/2874] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jun 30 12:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:44:56.192751 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:56.203571 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e40b25-ad00-4590-84bb-7706b029f39a admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:56.208493 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:56.209416 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] REQUEST_METHOD: `PUT` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.209693 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.209963 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] PATH_INFO: `/v3/projects/04570604c09c44a0be08f3729fa06e9e/users/afe61a071ba942c090e5b866743312d7/roles/bb98be56cb17479cb7d590fc8df81574` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.211992 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] RBAC: Authorizing `identity:create_grant(project_id=04570604c09c44a0be08f3729fa06e9e, user_id=afe61a071ba942c090e5b866743312d7, role_id=bb98be56cb17479cb7d590fc8df81574)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:56.214669 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:56.225791 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1435/2875] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jun 30 12:44:56 2022] PUT /identity/v3/projects/04570604c09c44a0be08f3729fa06e9e/users/afe61a071ba942c090e5b866743312d7/roles/bb98be56cb17479cb7d590fc8df81574 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:44:56.236246 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.236531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.236847 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.264255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] MFA Rules not processed for user `afe61a071ba942c090e5b866743312d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:56.265554 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:56.335577 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1441/2876] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:56.348552 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:56.350762 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:56.411219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1004196-3c95-43a4-9288-dd1b7911988c admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:56.417366 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3ea5f-e809-4010-881b-eb96e9309a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:56.418388 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3ea5f-e809-4010-881b-eb96e9309a29 service cinder] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.418795 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3ea5f-e809-4010-881b-eb96e9309a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.419175 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3ea5f-e809-4010-881b-eb96e9309a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.434965 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3ea5f-e809-4010-881b-eb96e9309a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:56.437079 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3ea5f-e809-4010-881b-eb96e9309a29 service cinder] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:56.447364 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1436/2877] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:56.527688 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:56.538600 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7553c-e2f6-45c4-b560-6b88d16b0dca None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:56.542449 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61b8631879a461db96399a9b91682c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:56.543146 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:56.543427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:56.543711 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:56.591918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:44:56.606582 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:56.608180 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:56.618430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1442/2878] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:44:58.813773 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:58.814263 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:58.814362 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:58.832443 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] MFA Rules not processed for user `6b952a9d6bad40c4b2829256bed0e017`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:44:58.833717 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:58.890008 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1437/2879] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:44:59.136926 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:44:59.138397 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:59.197515 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890ce2-847a-4b17-bfed-c4b962d7f27e service cinder] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:44:59.201193 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:44:59.201849 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:44:59.202127 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:44:59.202398 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:44:59.204290 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:44:59.214186 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:44:59.215762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fb403-bc38-4c1f-a3f3-8373a9bfc839 service neutron] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:44:59.223996 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1443/2880] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:44:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:45:01.382838 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:01.450291 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc81b8-e319-4f4d-a9ae-f810dde37748 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:01.455556 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:01.456451 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:01.456731 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:01.457028 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] PATH_INFO: `/v3/users/0520c4d9bbc24afebe791f7e152e9405` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:01.458276 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0520c4d9bbc24afebe791f7e152e9405)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:01.460219 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:01.474040 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:01.514038 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0520c4d9bbc24afebe791f7e152e9405'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:01.514862 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0520c4d9bbc24afebe791f7e152e9405'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:01.529589 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0520c4d9bbc24afebe791f7e152e9405'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:01.530292 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0520c4d9bbc24afebe791f7e152e9405'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:01.538783 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0520c4d9bbc24afebe791f7e152e9405'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:01.551921 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1438/2881] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:45:01 2022] DELETE /identity/v3/users/0520c4d9bbc24afebe791f7e152e9405 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:45:01.556419 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] REQUEST_METHOD: `POST` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:01.556932 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:01.557042 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:01.574461 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.auth.core [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] MFA Rules not processed for user `bfba4b29b3e04e04b163f6e8cff16484`. Rule list: `[]` (Enabled: `True`). {{(pid=80097) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:45:01.575677 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:01.636730 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1444/2882] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jun 30 12:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:45:01.647822 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:01.649790 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:01.714195 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7526-02e1-42a8-aee3-2dfc3fbd5ef1 admin admin] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:01.718174 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b91e199622457787fa2a7e78d743c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:01.718914 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:01.719193 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:01.719473 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:01.729450 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:01.730892 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:01.741525 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1439/2883] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:45:02.127084 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:02.128613 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:02.141594 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0accd-20ca-4076-9ef8-752020cd0425 None None] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:02.147495 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:02.148512 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:02.149263 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:02.149602 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] PATH_INFO: `/v3/projects/728a8430417f4faf8ce27400ab7a4c89` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:02.150918 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728a8430417f4faf8ce27400ab7a4c89)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:02.152931 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:02.203371 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728a8430417f4faf8ce27400ab7a4c89', 'request_id': 'req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.209691 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728a8430417f4faf8ce27400ab7a4c89', 'request_id': 'req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.215863 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728a8430417f4faf8ce27400ab7a4c89', 'request_id': 'req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.229457 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] The token cache is being invalidate because project 728a8430417f4faf8ce27400ab7a4c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:45:02.229746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.230391 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.231997 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1445/2884] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:45:02 2022] DELETE /identity/v3/projects/728a8430417f4faf8ce27400ab7a4c89 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:45:02.236510 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:02.247748 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a57bc9-813c-42cc-9fe5-b42dd256fd1a service neutron] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:02.309296 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:02.310204 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:02.310484 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:02.310756 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] PATH_INFO: `/v3/users/afe61a071ba942c090e5b866743312d7` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:02.312063 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe61a071ba942c090e5b866743312d7)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:02.314393 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:02.356498 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:02.367506 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe61a071ba942c090e5b866743312d7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.368158 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe61a071ba942c090e5b866743312d7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.380408 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe61a071ba942c090e5b866743312d7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.381013 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe61a071ba942c090e5b866743312d7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.388937 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-4e62ff78-5d6c-43cd-8cf0-1f62d4aef25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe61a071ba942c090e5b866743312d7'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.404875 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1440/2885] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:45:02 2022] DELETE /identity/v3/users/afe61a071ba942c090e5b866743312d7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:45:02.695629 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:02.697184 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:02.710801 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dcc39-b3e3-4f0b-b68a-812620bc6110 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:02.721240 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:02.773206 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:02.774445 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:02.774723 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:02.775031 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] PATH_INFO: `/v3/projects/04570604c09c44a0be08f3729fa06e9e` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:02.776228 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04570604c09c44a0be08f3729fa06e9e)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:02.778188 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:02.828448 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04570604c09c44a0be08f3729fa06e9e', 'request_id': 'req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.834519 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04570604c09c44a0be08f3729fa06e9e', 'request_id': 'req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.840744 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04570604c09c44a0be08f3729fa06e9e', 'request_id': 'req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.855620 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] The token cache is being invalidate because project 04570604c09c44a0be08f3729fa06e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80097) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:45:02.855905 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.856537 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:02.858145 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1446/2886] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:45:02 2022] DELETE /identity/v3/projects/04570604c09c44a0be08f3729fa06e9e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:45:12.401224 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:12.852531 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:18.843007 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72a59-3ce6-4144-a71b-8669d65b6702 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:18.843625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72a59-3ce6-4144-a71b-8669d65b6702 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:18.843625 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72a59-3ce6-4144-a71b-8669d65b6702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:18.872046 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-edc72a59-3ce6-4144-a71b-8669d65b6702 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:45:18.873270 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-edc72a59-3ce6-4144-a71b-8669d65b6702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:18.940495 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1441/2887] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jun 30 12:45:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:45:19.000565 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:19.002241 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:19.099355 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d81e81-6a10-4d05-9c70-4ddb4e33baa9 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:19.104866 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:19.105525 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:19.105809 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:19.106114 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:19.114776 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:19.116720 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:19.125529 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1447/2888] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:45:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:45:28.938092 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=31 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:29.122619 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=43 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:33.833807 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] REQUEST_METHOD: `POST` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:33.834318 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:33.834432 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:33.853009 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.auth.core [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] MFA Rules not processed for user `a4dec05937f84eedbbd5795582a16964`. Rule list: `[]` (Enabled: `True`). {{(pid=80096) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:45:33.854207 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:33.879340 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1442/2889] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jun 30 12:45:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:45:33.891768 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:33.910616 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca03171-30e0-481b-ab0a-04eddec7e826 service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:33.916129 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b96cbdec0f56412fb62617f03e77e713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:33.916813 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:33.917099 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:33.917374 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:33.926783 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:33.928414 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:33.937703 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1448/2890] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jun 30 12:45:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:45:35.873356 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:35.874852 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:35.952767 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6cc62-a48e-4cbc-9ed6-77718de193d6 None None] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:35.957980 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:35.958664 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] REQUEST_METHOD: `GET` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:35.958951 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:35.959227 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:35.961096 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.fernet_utils [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80096) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:35.974197 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:35.975741 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:35.987597 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1443/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:45:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:45:36.194427 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:36.196055 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:45:36.293888 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417563d-1199-4a52-a5c4-96b48cf316ee service swift] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:36.299541 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:36.300431 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:36.300746 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:36.301005 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] PATH_INFO: `/v3/users/6b952a9d6bad40c4b2829256bed0e017` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:36.302276 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b952a9d6bad40c4b2829256bed0e017)` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:36.304255 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:36.362988 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b952a9d6bad40c4b2829256bed0e017'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.363741 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b952a9d6bad40c4b2829256bed0e017'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.377655 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b952a9d6bad40c4b2829256bed0e017'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.378342 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b952a9d6bad40c4b2829256bed0e017'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.387164 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.notifications [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b952a9d6bad40c4b2829256bed0e017'} {{(pid=80097) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.401032 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1449/2892] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jun 30 12:45:36 2022] DELETE /identity/v3/users/6b952a9d6bad40c4b2829256bed0e017 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:45:36.590994 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] Authenticating user token {{(pid=80096) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:45:36.602087 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5fd2c-74ca-4e74-895a-3b99b41466af service placement] Validating token access rules against request {{(pid=80096) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:45:36.664382 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfba4b29b3e04e04b163f6e8cff16484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7a9e0309e534d4980ce851f75758fe8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80096) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:45:36.665290 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:45:36.665573 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:45:36.665847 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] PATH_INFO: `/v3/projects/b9ebc78f23044b98a13e5e3ebc5dd614` {{(pid=80096) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:45:36.667054 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9ebc78f23044b98a13e5e3ebc5dd614)` {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:45:36.669020 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] RBAC: Authorization granted {{(pid=80096) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:45:36.719698 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9ebc78f23044b98a13e5e3ebc5dd614', 'request_id': 'req-81192260-7262-4777-bb8e-d83b2fe09030'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.729869 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9ebc78f23044b98a13e5e3ebc5dd614', 'request_id': 'req-81192260-7262-4777-bb8e-d83b2fe09030'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.736206 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9ebc78f23044b98a13e5e3ebc5dd614', 'request_id': 'req-81192260-7262-4777-bb8e-d83b2fe09030'} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.743320 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:36.748689 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] The token cache is being invalidate because project b9ebc78f23044b98a13e5e3ebc5dd614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80096) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 12:45:36.749189 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.750286 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG keystone.notifications [None req-81192260-7262-4777-bb8e-d83b2fe09030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80096) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 12:45:36.752485 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: [pid: 80096|app: 0|req: 1444/2893] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jun 30 12:45:36 2022] DELETE /identity/v3/projects/b9ebc78f23044b98a13e5e3ebc5dd614 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:45:46.397256 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:45:46.753744 dv-cinder-846146-5-iscsi devstack@keystone.service[80096]: DEBUG dbcounter [-] [80096] Writing DB stats keystone:SELECT=1 {{(pid=80096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:47:03.987762 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Authenticating user token {{(pid=80097) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:47:03.989453 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:47:04.107671 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac5933-cdf1-4966-a6b4-2212de9f38c7 admin admin] Validating token access rules against request {{(pid=80097) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:47:04.174157 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0783ae884dd1434bbfc76b91457858f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a65694b54a64a0e8d83a436ed9184cf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80097) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:47:04.174823 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] REQUEST_METHOD: `GET` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:47:04.175083 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] SCRIPT_NAME: `/identity` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:47:04.175430 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80097) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:47:04.178658 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.fernet_utils [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80097) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:47:04.192921 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:47:04.194553 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8345092-7698-445b-bd63-a0159c7e191f service placement] RBAC: Authorization granted {{(pid=80097) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:47:04.205501 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: [pid: 80097|app: 0|req: 1450/2894] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jun 30 12:47:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:47:14.200972 dv-cinder-846146-5-iscsi devstack@keystone.service[80097]: DEBUG dbcounter [-] [80097] Writing DB stats keystone:SELECT=67 {{(pid=80097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}